Geographic view of a modelling system

ABSTRACT

Geo View is a three-dimensional virtual universe in which a real-world or virtual object may be represented by one or more virtual objects whose attributes are derived from attributes of the real-world object via a flexible user-specifiable mapping. Typically a two-dmensional plane located in three-dimensional space is used to visualise the universe of interest. The placement of virtual objects in the universe typically having a shape is governed by the absolute or relative geographical location of the real-world objects, and also by a flexible set of user-specified layout rules. In addition to the visualisation of various objects, the human observer can attach sounds to objects. The representation of real-world objects with rapidly time-changing attributes may be simplified by the use of Synthetic Strobes, flexible user-specified filters which changes in the visual attributes of a shape from one time-domain shift to another.

PART 1 SHAPES VECTOR

[0001] 1 Shapes Vector Introduction

[0002] Shapes Vector is the name given by the inventors to a particularcollection of highly versatile but independent systems that can be usedto make real world systems observable by a human operator. By providingan observation system the human may be able to detect using one or moreof their senses anomalies and the like in the real world system. Moreparticularly, the invention/s disclosed herein are in the field ofinformation observation and management.

[0003] To assist the reader, a particular combination of these elementsis described in an example. The example is in the field of computernetwork intrusion detection, network security management and eventsurveillance in computer networks. It will however be apparent to thoseskilled in the art that the elements herein described can exist andoperate separately and in different fields and combinations to that usedin the example.

[0004] The different system elements developed by the inventors are theresult of the use of several unusual paradigms that while separatelymake a their contribution also act synergistically to enhance theoverall performance and utility of the arrangement they form part of.

[0005] An embodiment in the computer network field is used to illustratean observation paradigm that works with a collection of elements, toprovide a near real-time way for observing information infrastructuresand data movement. The user (human observer) is provided sophisticatedcontrols and interaction mechanisms that will make it easier for them todetect computer network intrusion and critical security managementevents in real time as well as allow them to better analyse past events.The user may be computer assisted as will be noted where appropriate.

[0006] However, as stated previously each of the elements of the systemdisclosed herein are also capable of being used independently of theother. It is possible for each of them to be used in differentcombinations, alone or in conjunction with other elements as well asbeing the precursor for elements not yet created to suit a particularenvironment or application.

[0007] Whilst the Shapes Vector embodiment provided is primarily meantto aid computer intrusion detection, the system and or components of it,can be arranged to suit a variety of other applications, e.g data andknowledge mining, command and control, and macro logistics.

[0008] Shapes Vector is a development in which a number of keytechnologies have been created that include:

[0009] a high-performance multi-layer observation facility presentingthe user with a semantically dense depiction of the network underconsideration. To cater to the individual observational capacities andpreferences of user analysts, the specifics of the depiction are highlyuser-customable and allow use of more than just the users visual andmental skill;

[0010] a framework for “intelligent agents”; artificial intelligentsoftware entities which are tasked with co-operatively processingvoluminous raw factual observations. The agents can generate asemantically higher-level picture of the network, which incorporatessecurity relevant knowledge explicitly or implicitly contained withinthe raw input (however, such agents can be used to process other typesof knowledge);

[0011] special user interface hardware designed especially to supportDefensive Information Operations in which several user analysts operatein real-time collaboration (Team-Based Defensive InformationOperations).

[0012] an inferencing strategy which can coexist with traditionaldeductive mechanisms. This inferencing strategy can introduce certaintymeasures for related concepts.

[0013] The subject matter of this disclosure is complicated and it isboth a hindrance and a necessity to present particular elements of theShapes Vector system in the same document.

[0014] However, it will be apparent to those skilled in the art thateach element that makes up the Shapes Vector system is capable ofindependent existence and operation in different environments.

[0015] To reflect to some degree the independence of the elementsdisclosed, this specification is comprised of different parts that eachhave their own paragraph numbering but page numbering is consistent withtheir being included in a single document.

[0016] Part 1

[0017] Shapes Vector Introduction

[0018] Part 2

[0019] Shapes Vector Master Architecture and Intelligent AgentArchitecture

[0020] Part 3

[0021] Data View Specification

[0022] Part 4

[0023] Geo View Specification

[0024] Part 5

[0025] Tardis (Event Handler) Specification

[0026] A detailed index of the various parts and sections is provided onthe last pages of the specification to assist random access to theinformation provided herein or to make cross-referencing simpler.

[0027] Part 1 is an overview of the Shapes Vector embodiment thatdescribes a particular environment and discloses in a general way someof the elements that make up the total system. Parts 2, 3, 4 and 5disclose fundamental aspects of the Intelligent Agent Architecture, DataView, Geo View and the Tardis (Event Handler) specificationrespectively, terms that will be more familiar once the specification isread and understood.

[0028] This patent specification introduces the Shapes Vector system byfirstly describing in Sections 1 and 2 of Part 1, the details of itstop-level architecture. Included are details of the hardware andsoftware components present in a system presently under construction.Section 3 of Part 1, gives an overview of the first set of observation(some times referred to as visualisation) paradigms, which have beenincorporated into the system. Two different views of computer/telecommunications networks are described in this section, bothpresenting a three-dirnensional “cyberspace” but with vastly differentapproaches to the types of entities modelled in the space and how theyare positioned (and dynamically repositioned). Some preliminary commentsare offered as to the effectiveness of one of these views, “Geo View”,for network defence. “Geo View” is another of those terms that will bebetter understood after a reading of the document.

[0029] A description of the intelligent agent architecture follows inSection 4 of Part 1, including an overview of the multi-layered ShapesVector Knowledge Architecture (SVKA) plus details of the inferencingstrategies. The knowledge processing approach is very general, and isapplicable to a wide variety of problems. Sections 5 and 6 of Part 1describe special techniques employed within the Tardis (Event Handling)system to assist a user analyst to observe the time-varying behaviour ofa network. Two principal mechanisms are detailed, Synthetic Strobes andSelective Zoom, along with some hypotheses as to how such mechanismsmight be extended to offer even greater flexibility. Section 7 of Part 1of the patent specification details a comparative analysis of relatedresearch and a set of conclusions summarising the broad thrusts of theShapes Vector system.

[0030] More detailed disclosures of these elements of the invention areprovided in Parts 2, 3, 4 and 5.

[0031] In reading this specification, it should be noted that while someissues are dealt with in detail, the specification is also used todisclose as many of the paradigms and strategies employed as possible,rather than discussing any one paradigm in depth. In an attempt toprovide an example of how these paradigms and strategies are used,several new mechanisms for dealing with information in a real-timeenvironment are described in the context of the information securityfield but in no way are the examples meant to limit the application ofthe mechanisms revealed.

[0032] Observation is a term used in this specification to embody theability of a human to observe by experience through a variety of theirsenses. The senses most used by a human user include sight, hearing andtouch. In the embodiment and system developed thus far all of thosesenses have been catered for. However, the term observe is not used inany limiting way. It may become possible for a human's other senses tobe used to advantage not only in the scenario of computer systemsecurity but others within the realm of the imagination of the designerusing the principles and ideas disclosed herein. A human could possiblyusefully use their other senses of smell, taste and balance inparticular future applications.

[0033] In this specification the term clients is used to refer to asource of events based on real and virtual objects operating in the realworld and the term monitors is used to refer to one or more recipientsystems that make the events observable to a human user.

[0034] The following discussion will provide background informationrelating to the described embodiment of the invention or existingparadigms and strategies and when it does so it is intended purely tofacilitate a better understanding of the invention/s disclosed herein.However, it should be appreciated that any discussion of backgroundinformation is not an acknowledgment or admission that any of thatmaterial was published, known or part of the common general knowledge asat the filing date of the application.

[0035] 2 Architectural Components

[0036] 2.1 Primary Functional Architecture

[0037] At the coarsest level, the Shapes Vector system can be consideredto be composed of a series of “macro-objects,” shown in FIG. 1. Thesemodules interact with one another in various ways: the lines in thefigure indicate which objects interact with others. The functionsperformed by each of these macro-objects and the purpose and meaning ofthe various inter-object interactions are described in the parts andsections that follow.

[0038] 2.1.1 Configuration Interface and I/O Sub-system

[0039] The Configuration Interface and I/O macro-objects collectivelyencapsulate all functionality, involving interaction with the user ofthe Shapes Vector system. They in turn interact with the Display, Tardis(Event Management) and Intelligent Agent macro-objects to carry out theuser's request. In addition to being the point of user interaction withthe system, this user-interface macro-object also provides the abilityto customise this interaction. Refer to FIG. 1, which displays theFunctional Architecture of Shapes Vector. A user can interactivelyspecify key parameters, which govern the visual and other environmentsgenerated by Shapes Vector and the modes of interaction with thoseenvironments. Such configurations can be stored and retrieved acrosssessions allowing for personal customisation.

[0040] Individual users can set up multiple configurations for differentroles for which they might wish to use the system. Extensive undo/redocapabilities are provided in order to assist with the investigation ofdesired configurations.

[0041] The observation of the Shapes Vector world is user-customable bydirect interaction with a structure called the “Master Table” (seeSection 3). In this table the user can in one example, associate visualattributes, such as shape, colour and texture, with classes of objectsand their security-relevant attributes.

[0042] A user interacts with the Shapes Vector system via any number ofinput and output devices, which may be configured according to eachindividual user's preferences. The input devices may be configured at adevice-specific level, for example by setting the acceleration of atrackball, and at a functional level, by way of further example, byassigning a trackball to steer a visual navigation through a3-dimensional virtual world representative of a computer network. TheAppendix to Part 1 describes the typical user interface hardwarepresented to a Shapes Vector user.

[0043] 2.1.2 Sensors

[0044] Sensors can take many forms. They can be logical or physical. Atypical example would be an Ethernet packet sniffer set to tap rawpackets on a network. In another example, the sensor can be the outputof a PC located at a remote part of a network, which undertakespre-processing before sending its readings of itself or the network backto the main Shapes Vector system components. Other examples are Softwareor Hardware to capture packets in a digital communication network, toexamine the internal or operating state of a computer or to analyseaudit records created by a computer of network device. Sensors transmittheir data into the level one portion of the Intelligent Agent Gestalt(this term will also have more meaning after further reading of thespecification) for further processing. Some of the processing involvedcould entail massaging of data for Knowledge Base storage, or perhapssimple logical deductions (first order logic facts).

[0045] 2.1.3 Intelligent Agent Architecture

[0046] 2.1.3.1 Knowledge Base

[0047] The Knowledge object is essentially a knowledge base containingfacts about the overall domain of discourse relevant to Shapes Vector.The knowledge is represented in terms of context-free Entities andRelationships, allowing for its efficient storage in a relationaldatabase. Entities constitute not only physical devices such ascomputers and printers, but also logical objects such as files anddirectories. Each entity possesses a set of security-relevantattributes, which are stored within the knowledge base. For each storedobservation of an entity attribute, there is accompanying meta-data thatincludes the time of discovery, which agent or sensor discovered it andan expiry time for the data. The current knowledge base models severaltypes of inter-entity relationships, including physical connectivity,physical or logical containment, bindings between processors andprocesses, roles of processes in client-server communications, originand destination of packet entities, and so on.

[0048] 2.1.3.2 Intelligent Agents and Ontologies

[0049] The Intelligent Agent macro-object encapsulates the artificialintelligence aspects of the Shapes Vector system. It specificallyincorporates a (potentially very large) family of intelligent agent,software entities imbued with expert knowledge in some particular domainof discourse over which they may make deductions. Agents within theShapes Vector systems are arranged into a series of “abstraction layers”or “logical levels” with each agent existing at only one such layer.Agents operate by accepting knowledge of a particular abstraction,possibly from several sources in lower layers, and generating newknowledge of a higher level of abstraction through a deductive process.An agent that resides at layer n of the Shapes Vector KnowledgeArchitecture must receive its input knowledge in the form of assertionsin a knowledge representation known as the “Level n Shapes Vectorontology”. Any deductive product from such an agent is expressed interms of the (more abstract) “Level n+1 Shapes Vector ontology”.

[0050] Entities in the Intelligent Agent macro-object can be broken intocategories: data-driven entities and goal-driven entities. The formergroup is characterised by a processing model wherein all possiblecombinations of input facts are considered with an eye towardsgenerating the maximum set of outputs. A common method employed beingforward chaining. Goal-driven entities adhere to a different executionmodel: given a desirable output, combinations of inputs are considereduntil that output is indicated, or all combinations are exhausted.

[0051] Intelligent Agents and the goals and functionality of the ShapesVector Knowledge Architecture are covered in more depth in Section 4 ofthis part of the specification and in Part 2 of the specification.

[0052] 2.1.4 The Tardis

[0053] The Tardis is a real-time event management system. Its task is toschedule and translate the semantic deductions from Intelligent Agentsand sensors into events capable of being visualised by the displaymodule or sub-system. The Tardis also encapsulates the Shapes Vectorsystem's notion of time. In fact, the operator can shift the systemalong the temporal axis (up to the present) in order to replay events,or undertake analyses as a result of speeded-up or slowed-down notionsof system time.

[0054] 2.1.5 Monitor

[0055] Monitor preferably renders three-dimensional (3D) views ofobjects and their interactions in real-time. As can be seen, there are anumber of basic views defined all of which can be navigated. Eachdifferent view is based on a fundamental visualisation paradigm. Forexample, Geo View is based on location of virtual objects within aspace-time definition, whereas Data View's location of virtual objectswithin its space is based on the data interaction.

[0056] Several reusable modules make up the composition of each view.These include elements such as data structures identifying the shapes,textures, and visual relationships permitted for each class of object,as well as common rendering methods for representing the view'sUniverse.

[0057] The paradigms for some of the views are discussed in more detailin later sections. It will be appreciated that the visualisationparadigms are in fact specific embodiments of the observationalrequirement of the system, wherein a human user can use one or more oftheir senses to receive information, that could include aural and hapticinteraction.

[0058] 2.2 The Hardware

[0059] In a preferred embodiment of this invention, the hardwarearchitecture of the Shapes Vector system consists of a primary serverequipped with a powerful computational engine and high-performance 3Dgraphics capabilities, a database server, a dedicated 100BaseT Ethernetnetwork, one PC with specialised 3D audio hardware, and one PC with userinput devices attached. A preferred configuration is shown schematicallyin FIG. 2.

[0060] The preferred observational environment of the Shapes Vectorworld can be rendered in 3D stereo to provide aural information andpreferably viewed using Crystal Eyes™ shutter glasses synchronised tothe display to provide purely visual information. Crystal Eyes™ waschosen for visualisation, as this product allows the user to be immersedin a 3D world on a large screen while still permitting real worldinteraction with fellow team-members and the undertaking of associatedtasks, e.g. writing with a pencil and pad, that are features notavailable with head-mounted displays.

[0061] In addition to 3D graphics capabilities, there is a soundrendering board, which is used to generate multi-channel real-time 3Daudio. Both the 3D graphics and sound rendering board make use of headtracking information in producing their output. The graphics renderermakes use of tracking information to alter the perspective of thedisplayed world so that the user experiences the effect of moving arounda fixed virtual world. The sound renderer makes use of head movementtracking information to alter the sound-scape so that the user alsoexperiences the effect of moving around in a fixed world with relevantsounds. That is, where a particular sound source will be perceived to becoming from the same fixed place irrespective of the users headmovement. The perception of direction in 3D sound is enhanced by theability to turn one's head and listen. For instance, it is oftendifficult to determine whether a sound is coming from in front or behindwithout twisting one's head slightly and listening to determine in whichear a sound is received first or loudest. These perceptive abilities aresecond nature to humans and utilisation of them is a useful enhancementof the information presentation capabilities of Shapes Vector.

[0062] A joystick and rudder pedals preferably provide the primary meansof navigation in the 3D world. User input to the system is to beprovided primarily through the touch screen and via voice recognitionsoftware running on a PC. Haptic actuators are realisable using audiocomponents to provide a feeling of say roughness as the user navigatesover a portion of the virtual world. Many other actuators are possibledepending on the degree of feedback and altering required by the user.

[0063] The initial prototype of Shapes Vector had the user input/outputdevices connected to a workstation or PC with software connecting theremote peripherals with the User Interface proper. The layout of theShapes Vector workstation (ie, the physical arrangement of the userinterface hardware) will vary depending upon the operational role andthe requirements of individual users, as described in the Appendix toPart 1 of the specifcation.

[0064] 2.3 System Software

[0065] In the embodiment described herein Shapes Vector is implementedas a distributed system with individual software components thatcommunicate between each other via TCP/IP sockets. A simple customprotocol exists for encoding inter-process communication. To limitperformance degradation due to complex operating system interaction, thesystem processes are used only for relatively long-lived elements ofcontrol (e.g. the knowledge base server, or an intelligent agent).Shorter-lived control is implemented through threads.

[0066]FIG. 3 indicates where the primary software modules will berunning in the initial system as well as a schematic of the hardwaremodules they are associated with. While most of the implementation ofthe Shapes Vector system has been custom-coded, the system does make useof a number of different software technologies to supply servicefunctionality. Intelligent Agents make extensive use of NASA's CLIPSsystem as a forward chaining engine, and also use Quintus Prolog™ toimplement backward chaining elements. Additionally, the knowledge baseand its associated servers are preferably implemented using the Oracle™relational database management system.

[0067] The graphics engine of the Display macro-object is preferablybuilt upon an in-house C++implementation of the Java 3D API and utilisesOpenGL™ for the low-level rendering. The User Interface elements arebuilt using Sun Visual Workshop™ to produce X Windows Motif™ GUIelements.

[0068] 3 The “Classical” Visualisation Paradigm

[0069] The classical visualisation paradigm refers to methods that arederived from mechanisms such as geographic layout, and relatively staticrules for objects. While some may not regard what is described here asentirely “classical”, it serves to distinguish some of the visualisationmethods from the relatively more “bizarre” and therefore potentiallymore interesting visualisation paradigms described in thisspecification.

[0070] Using by way of example information security as the environmentto be modelled and observed the fundamental basis of the classicalvisualisation paradigm is to associate a security-relevant attributewith a visual entity or a visual property of an entity, eg. shape,colour, or texture.

[0071] A Shapes Vector hypothesis is that any visualisation paradigm isnot only “sensitive” to its application, ie. some paradigms are bettersuited fo specific classes of application, but that the implementationof the paradigm is sensitive to the specific user. It is thus claimedthat not only should a visualisation system be customable to take intoaccount the type of application, but also it must have highly customablefeatures to take into account individual requirements and idiosyncrasiesof the observer. That is, the customisability of the system is veryfine-grained.

[0072] In fine grained customable systems, it is important that journalrecords and roll-back facilities are available in the certain knowledgethat users will make so many changes that they will “lose” their way andnot be sure how to return to a visual setting they find more optimalthan the one they are currently employing.

[0073] In an embodiment, users can associate attributes to shapes,colour, texture, etc. via manipulation of a master table, whichdescribes all visual entities (with security-relevant attributes) thesystem is able to monitor. This table contains user-customabledefinitions for shapes, colours, and textures employed in thevisualisation of the entity. For example, the security attribute “readenable” can be associated with different colours, transparencies ortextures. Part of the essence of Shapes Vector involves utilising thevisualization process as a method for users to divine (via inductiveinference) patterns in the “security cyberspace”. These patterns have anattached semantic. Typically, we expect users to note anomalies from themyriad system activities that represent authorised use of the system.Given these anomalies, the user will be able to examine them moreclosely via visualisation, or bring into play a set of IntelligentAgents to aid an in depth analysis by undertaking deductive inference.

[0074] Not withstanding the above, there is also a semantic gap betweenwhat an Intelligent Agent can deduce and what a user can discern usingtheir senses. The approach in this embodiment is based on the hypothesisthat in most cases the observational interface element will be employedfor highlighting macro matters, while the agents will focus on micromatters. These micro deductions can be fed to the visualisation engineso that a user can observe potential overall state changes in a system,thereby permitting a user to oversee and correlate events in very largenetworks.

[0075] 3.1 Geo View

[0076] Geo View is perhaps the most classical of the visualisationparadigms. Its basis is a two dimensional plane located inthree-dimensional space. The plane represents the traditional geographicplane: location in the virtual plane represents the physical location ofobjects. FIG. 4 is a depiction of a small network where the primarypoints of interest involve a set of computers and the data that isflowing between them. The sizes, shape, and texture of objects all carryan associated semantic. The double pyramid shapes with a third pyramidembedded at the top are representative of computers with networkinterfaces. Also quite visible is the packet flow between the computersin the star network. Although not explained here, to the trained eye thestart of a telnet session, some web traffic, as well as X Windowselements is also represented.

[0077] The Shapes Vector system permits a user to select classes ofobjects and render them above the plane. In fact it is possible torender different classes of objects at different levels above or belowthe geographic base plane. This rendering tactic allows a user to focuson objects of interest without losing them in the context of the overallsystem. This “selective zoom” facility is described further in Section5.2 of this part.

[0078]FIG. 5 depicts a scene inside a machine object. In this view, twoprocessors each with several processes are depicted. In an animated viewof this scene the amount of processing power each of the processes isconsuming is represented by their rate of rotation. Again, the size,texture, and specific aspects of their shape can and are used to depictvarious semantics.

[0079] The transparent cube depicts a readable directory in which iscontained a number of files of various types.

[0080] In addition to the visualisation of various objects, the humanobserver can attach sounds and possibly haptic characteristics toobjects. In particular, the system is capable of compiling a “soundsignature” for an object (e.g. a process) and plays the resulting soundthrough speakers or headphones. This facility is quite powerful whendetecting event changes that may have security significance. Indeed, ina concept demonstrator, a change in the code space of a process causes adistinct change in its sound. This alerts the user when listening to aprocess (e.g. printer daemon) with a well-known characteristic soundthat something is not quite right. By inspecting the process visually,further confirmation can be forthcoming by noting that itscharacteristic appearance, e.g. colour, has changed. The use of hapticattributes can also be advantageous in certain circumstances.

[0081] One of the major issues that arise out of Geo View other than thebasic geographic location of nodes, is the structural relationship ofobjects contained in a node. For example, how does one depict thestructural relationship of files? FIG. 5 gives some indication of apreferred view in a directory containing files and possibly furtherdirectories is rendered in a particular way. In a system such as UNIX,there is an well-understood tree structure inherent in its file system.In other operating systems, the structure is not so precise. In thedescription so far, Geo View still lacks a level of structuralintegrity, but it must be realised that any further structure, which isimposed, may invalidate the use of the view for various applications orspecific user requirements.

[0082] Shapes Vector avoids some of the problems posed above byproviding a further level of customisation by permitting a user tospecify the structural relationship between classes of objects from apredetermined list (e.g. tree, ring). A run-time parser has beenconstructed to ensure that any structural specification must satisfycertain constraints, which guarantee that “nonsensical”, or circularrelationships, which are impossible to display, are not introduced.

[0083] 1. Geo View is a three-dimensional virtual universe in which areal-world or virtual object may be represented by one or more virtualobjects whose visual attributes are derived from attributes of thereal-world object via a flexible user-specifiable mapping (called hereina “Master Table”). The placement of virtual objects typically having ashape within the universe is governed by the absolute or relativegeographical location of the real-world object, and also by a flexibleset of user-specified layout rules. Layout rules permit thespecification of a structured layout for groups of shapes whosereal-world objects and virtual objects have some commonality. The listof structures includes, but is not limited to linear, grids, star, ringand graph.

[0084] 2. Changes to the visual attributes of shapes (e.g., size orheight above a plane) may be made dynamically by a user (humanobserver). Such changes may be applied to all shapes in the universe orto those which match user-specified criteria. This facility is termedherein “Selective Zoom”.

[0085] 3. The user may configure Audio cues (sounds and/or voices) todenote the attributes of represented objects (through a Master-Tableconfiguration), or to denote the occurrence of a real-world event. Suchcues may be associated with a point in three-dimensional space (i.e.,positional sound), or they may be ambient.

[0086] 4. The representation of real-world objects with rapidlytime-changing attributes may be simplified by the use of SyntheticStrobes, flexible user-specified filters which shift changes in thevisual attributes of a shape from one time-domain to another. SyntheticStrobes may be applied across the entire universe or selectivelyaccording to a flexible user-specification. Such strobes may also beused to shift slow changes in the attributes of a shape into a fasterdomain (e.g., so that a human may perceive patterns in very slowlyaltering real-world objects).

[0087] 5. A user may select shapes within a Geo View universe (eitherinteractively or by a flexible user-specified condition) and choose tohave the corresponding set of shapes in another view (e.g., a Data Viewor a different Geo View) highlighted in a visual manner. Thespecification of the condition defining correspondence of shapes betweenuniverses may be made in a flexible user-defined fashion.

[0088] A user may also specify structural arrangements to be used by GeoView in its layout functions. For example, “located-in”, “in-between”,and “attached-to” are some of the operators available. These allow aflexible layout of shapes and objects preserving user requiredproperties without requiring specific coordinates being supplied for allobjects.

[0089] 3.2 Data View

[0090] A problem with Geo View is that important events can be missed ifheavily interacting objects or important events are geographicallydispersed and not sufficiently noticeable. In Section 5 of this part, wediscuss mechanisms that can be utilised to avoid this problem in somecircumstances. However, in this section we describe a preferred viewthat is also intended to address parts of this problem. Parts 3 and 4 ofthe specification provides a more detailed account of this approach.

[0091] Geo View has its roots in depicting actions and events that havephysical devices and their location as an overriding theme. Of courselogical entities are shown, but again they have a geographic theme. DataView, as its name suggests, is intended to provide a view where thebasic paradigm is simply one of data driven events (eg. byte transfer)rather than geographic location. Heavily interacting objects, eg.producers and consumers of data, can be depicted as being located “closetogether”. Unlike Geo View, where the location of an object tends to berelatively static during its lifetime (copying of files is simply aspecial case of bringing a new object into existence) interaction anddata transfer between objects in Data View may be more dynamic. Thus,the location of objects is expected to be more dynamic. Therefore, rulesare preferred so as to define the layout of objects not only from theperspective of whether interaction occurred, but also the amount ofinteraction, and the rate of interaction;

[0092] It is intended in a preferred embodiment to utilise Newtoniancelestial mechanics and model interaction as forces on the interactionof objects as fundamental rules for the data view layout.

[0093] Each object has a mass that is based on its “size” (size is userdefined eg. the size of a file or code in a process). User definedinteraction between objects causes the equivalent of an electric chargeto build. This charge is attractive, whereas “gravity” resulting frommass is repulsive. The build-up of charge tends to negate the force ofgravity thereby causing objects to move closer together until some formof equilibrium is reached. Of course we need to adjust the basic Coulomband Newton's laws in order for the forces to balance appropriately. Todo so, we are lead to set axiomatically several calibration points. Thatis, we must decide axiomatically some equilibrium points; e.g. twoobjects of identical mass are in equilibrium X units apart with Y bytesper second flowing between them. Without these calibration points, thedistance and motion of the objects may not provide optimal viewing.Further to this requirement, it can be inferred that the force formulaemust be open to tinkering on a per user basis in order to permit eachuser to highlight specific interactions based on higher semanticsrelated to the user's security mission. A further rule, which ispreferred in this embodiment, is the rate of “decay” of charge on anobject. Otherwise, interacting objects will simply move closer andcloser together over time. This may be appropriate for some types ofvisual depiction for a user, but not for others. For example, retainedcharge is useful for a user to examine accumulative interaction over atime slice, but charge decay is a useful rule when examining interactionrates over a given time period.

[0094] The interaction mechanism described herein serves to indicate thebasis for interaction between objects and their location in space toprovide visual depiction of objects and their clusters for examinationby a user in order to arrive at inductive hypotheses.

[0095]FIG. 6 shows how Data View might visualise a collection ofdata-oriented objects (eg. files and/or servers) which interact with oneanother to varying degrees. Despite using proximity to show whether anobject is interacting with another, further visual mechanisms are neededfor the user to be able to analyse the type of data interaction, and thecurrent state of affairs of interaction within a specified time slice.Hence we still need visual markers which directly link one object toanother, for example an open socket connection between two processes,which actually has data in transit. These objects could initially bevery far apart due to previous low interaction status. However, sincethey are now interacting a specific connection marker may be needed tohighlight this fact. Given the type of interaction, the force formulaemay be adjusted so as to provide a stronger effect of interaction.However, this mechanism is restricted to classes of objects and theinteraction type, whereas the user may be particularly interested ininteraction between two particular object instances. Hence a visualmarker link would be more appropriate. Yet, one can imagine thecomplexity of a view if all markers are shown simultaneously. Henceactual connection lines, their size, shape, colour, motion and location,may be switched on and off via a set of defined criteria.

[0096] As for Geo View, Data View in its preferred embodiment, will comewith its own Master Table describing shapes and textures for variousattributes, as well as an input mechanism to describe relationshipsbetween objects based on a series of interaction possibilities. Theobjects presented in Data View may in some cases be quite different fromthose found in Geo View, while in other cases they will be similar oridentical. Clearly the defining difference lies in the fact that DataView's Master Table will focus less on physical entities and moreclosely on logical entities and data driven events.

[0097] Thus the preferred main features of Data View are as follows:

[0098] 1. A set of one or more two-dimensional virtual universes inwhich a real-world object may be represented by one or more shapes whosevisual attributes are derived from attributes of the real-world objectvia a flexible user-specifiable mapping (called a “Master Table”). Inone embodiment each universe is represented as a disc in a plane. Theplacement of a shape within a universe is governed by degree ofinteraction between the represented object and other objects representedin that universe. As an alternative, the view may be constructed as aset of one or more three-dimensional virtual universes with similarproperties.

[0099] 2. Interaction between a pair of real-world objects causes thepair of shapes that represent them to be mutually attracted. Themagnitude of this force is mathematically derived from the level ofinteraction. Real world Objects which interact are furthermore mutuallyrepelled by a “gravitational force”, the magnitude of which is derivedfrom attributes of the real-world objects in a flexible user-specifiedmanner. In one embodiment all forces are computed as vectors in theplane of the universe. The velocity of a shape in the universe isproportional to the vector sum of the forces applied to the shape (i.e.,in this embodiment there is no concept of acceleration).

[0100] 3. Shapes within a universe may be tagged with what is termedherein a “flavor” if their real-world object's attributes match aflexible user-specified condition associated with that flavor. A pair ofshapes may only attract or repel one another if they share one or moreflavors.

[0101] 4. Each shape within a universe maintains an explicit list ofother shapes it “interacts” with. A pair of shapes may only attract orrepel one another if each is in the interaction set of the other.

[0102] 5. Each shape within a universe may have a “radius of influence”associated with it, a user-specified region of the universe surroundingthe shape. A shape may only exert a force onto another shape if thelatter is within the radius of influence of the former. The radius ofinfluence of a shape may be displayed visually. The selection of whichshapes in the universe have radii of influence, and which of those radiishould be displayed, may be either universal or by means of a flexibleuser-specified condition.

[0103] 6. Each shape within a universe may optionally be visually linkedto one or more shapes in a different universe by a “Marker” whichrepresents a relationship between the real-world objects represented bythe shapes. The selection of which shapes in which universes should beso linked is by means of a flexible user-specified condition.

[0104] 7. Changes to the visual attributes of shapes (e.g., size orheight above a plane) may be made dynamically by a user. Such changesmay be applied to all shapes in the universe or to those which matchuser-specified criteria. This facility is termed “Selective Zoom”.

[0105] 8. The user may configure Audio cues (sounds and/or voices) todenote the attributes of represented objects, or to denote theoccurrence of a real-world event. Such cues may be associated with apoint in three-dimensional space, or they may be ambient.

[0106] 9. The representation of real-world objects with rapidlytime-changing attributes may be simplified by the use of SyntheticStrobes, flexible user-specified filters which shift changes in thevisual attributes of a shape from one time-domain to another. SyntheticStrobes may be applied across the entire universe or selectivelyaccording to a flexible user-specification. Such strobes may also beused to shift slow changes in the attributes of a shape into a fasterdomain (e.g., so that a human may perceive patterns in very slowlyaltering real-world objects).

[0107] 10. A user may select shapes within a Data View universe (eitherinteractively or by a flexible user-specified condition) and choose tohave the corresponding set of shapes in another view (e.g., a Geo Viewor a different Data View) highlighted in a visual manner. Thespecification of the condition defining correspondence of shapes betweenuniverses may be made in a flexible user-defined fashion.

[0108] 4 Intelligent Agents

[0109] Shapes Vector can utilise large numbers of Intelligent Agents(IA's), with different domains of discourse. These agents makeinferences and pass knowledge to one another in order to arrive at a setof deductions that permit a user to make higher level hypotheses.

[0110] 4.1 Agent Architecture

[0111] In order to achieve knowledge transfer between agents which isboth consistent and sound, ontology becomes imperative. The task ofconstructing a comprehensive ontology capable of expressing all of thevarious types of shapes is non-trivial. The principal complication comesfrom the fact that the structural elements of the ontology must becapable of covering a range of knowledge ranging from the very concrete,through layers of abstraction and ultimately to very high-levelmeta-knowledge. The design of a suite of ontological structures to coversuch a broad semantic range is problematic: it is unlikely to produce atidy set of universal rules, and far more prone to produce a complexfamily of inter-related concepts with ad hoc exceptions. More likely,due to the total domain of discourse being so broad, ontology producedin this manner will be extremely context sensitive, leading to manypossibilities for introducing ambiguities and contradictions.

[0112] To simplify the problem of knowledge representation to a pointwhere it becomes tractable, the Shapes Vector system chooses to define asemantic layering of its knowledge-based elements. FIG. 7 shows thebasic structure of this knowledge-architecture and thus the primaryarchitecture of the set of Intelligent Agent's (Al's). At the verybottom of the hierarchy are factual elements, relatively concreteobservations about the real world (global knowledge base). Factualelement can draw upon by the next layer of knowledge elements: thesimple intelligent agents. The communication of factual knowledge tothese simple knowledge-based entities is by means of a simple ontologyof facts (called the Level 1 Shapes Vector ontology). It is worthwhilenoting that the knowledge domain defined by this ontology is quiterigidly limited to incorporate only a universe of facts—no higher-levelconcepts or meta-concepts are expressible in this ontology. Thissimplified knowledge domain is uniform enough that a reasonably cleanset of ontological primitives can provide a concise description. Also,an agent may not communicate with any “peers” in its own layer. It mustcommunicate with a higher agent employing higher abstraction layerontology. These higher agents may of course then communicate with a“lower agent”. This rule further removes the chance of ambiguity andontology complexities by forcing consistent domain restrictedOntologies.

[0113] An immediate and highly desirable consequence of placing theseconstraints on the knowledge base is that it becomes possible torepresent knowledge as context free relations. Hence the use ofrelational database technology in storage and management of knowledgebecomes possible. Thus, for simple selection and filtering procedures onthe knowledge base we can utilise well known commercial mechanisms whichhave been optimised over a number years rather than having to build acustom knowledge processor inside each intelligent agent. Note that weare not suggesting that knowledge processing and retrieval is notrequired in an IA, but rather that by specifying certain requirements ina relational calculus (SQL preferably), the database engine assists usby undertaking a filtering process when presenting a view for processingby the IA. Hence the IA can potentially reap considerable benefits byonly having to process the (considerably smaller) subset of theknowledge base which is relevant to the IA. This approach becomes evenmore appealing when we consider that the implementation of choice forIntelligent Agents is typically a logic language such as Prolog. Suchenvironments may incur significant processing delays due to the heavystack based nature of processing on modern Von Neumann architectures.However, by undertaking early filtering processes using optimisedrelational engines and a simple knowledge structure, we can minimise thetotal amount of data that is input into potentially time-consuming treeand stack based computational models.

[0114] The placement of intelligent agents within the various layers ofthe knowledge hierarchy is decided based upon the abstractions embodiedwithin the agent and the knowledge transforms provided by the agent. Twocriteria are considered in determining whether a placement at layer n isappropriate:

[0115] would the agent be context sensitive in the level n ontology? Ifso, it should be split into two or more agents.

[0116] does the agent perform data fusion from one or more entities atlevel n? If so, it must be promoted to at least level n+1 (to adhere tothe requirement of no “horizontal” interaction)

[0117] Further discussion on intelligent agents and ontological issuescan be found elsewhere in the specification.

[0118] 4.2 Inferencing Strategies

[0119] The fundamental inferencing strategy underlying Shapes Vector isto leave inductive inferencing as the province of the (human) user anddeductive inferencing as typically the province of the IA's. It isexpected that a user of the system will examine deductive inferencesgenerated by a set of IA's, coupled with visualisation, in order toarrive at an inductive hypothesis. This separation of duties markedlysimplifies the implementation strategies of the agents themselves.Nevertheless, we propose further aspects that may produce a verypowerful inferencing system.

[0120] 4.2.1 Traditional

[0121] Rule based agents can employ either forward chaining or backwardchaining, depending on the role they are required to fulfil. Forexample, some agents continuously comb their views of the knowledge basein attempts to form current, up to date, deductions that are as “highlevel” as possible. These agents employ forward chaining and typicallyinhabit the lower layers of the agent architecture. Forward chainingagents also may have data stream inputs from low level “sensors”. Basedon these and other inputs, as well as a set of input priorities, theseagents work to generate warnings when certain security-significantdeductions become true. Another set of agents within the Shapes Vectorsystem will be backward chaining (goal driven) agents. These typicallyform part of the “User Avatar Set”: a collection of knowledge elementswhich attempt to either prove or disprove user queries.

[0122] 4.2.2 Vectors

[0123] While the traditional approach to inferencing is sufficient forsimple IA's which deal principally in the domain of concrete fact, it isless suitable for agents (typically from higher layers) which must dealwith uncertain and/or incomplete information. Typically, such agentsoperate in a more continuous knowledge domain than that underlyingrule-based deductive inferencing, and as such are not easily expressedin either a purely traditional forward or backward chaining paradigm.For these higher level agents, we instead make use in this embodiment ofan alternative inferencing strategy based upon notions of vector algebrain a multi-dimensional semantic space. This alternative strategy isemployed in conjunction with more conventional backward chainingtechniques. The use of each of the paradigms is dependent on the agent,and the domain of discourse.

[0124] Our vector-based approach to inferencing revolves aroundconstructing an abstract space in which relevant facts and deductionsmay be represented by geometrical analogues (such as points andvectors), with the proper algebraic relationships holding true. Ingeneral, the construction of such a space for a large knowledge domainis extremely difficult. For Shapes Vector, we adopt a simplifyingstrategy of constructing several distinct deductive spaces, each limitedto the (relatively small) domain of discourse of a single intelligentagent. The approach is empirical and is only feasible if each agent isrestricted to a very small domain of knowledge so that construction ofits space is not overly complex.

[0125] The definition of the deductive space for an IA is a methodicaland analytical process undertaken during the design of the agent itself.It involves a consideration of the set of semantic concepts (“nouns”)which are relevant to the agent, and across which the agent's deductionsoperate. Typically this concept set will contain elements of the agent'slayer ontology as well as nouns which are meaningful only within theagent itself. Once the agent's concept set has been discovered, we canidentify within it a subset of ‘base nouns’—concepts which cannot bedefined in terms of other members of the set (This identification isundertaken with reference to a semi-formal ‘connotation spectrum’ (acomparative metric for ontological concepts).

[0126] Such nouns have two important properties:

[0127] each is semantically orthogonal to every other base noun, and

[0128] every member of the concept set which is not a base noun can bedescribed as a combination of two or more base nouns.

[0129] Collectively, an IA's set of n base nouns defines ann-dimensional semantic space (in which each base noun describes anaxis). Deductions relevant to the agent constitute points within thisspace; the volume bounded by spatial points for the full set of agentdeductions represents the sub-space of possible outputs from that agent.A rich set of broad-reaching deductions leads to a large volume of thespace being covered by the agent, while a limited deduction set resultsin a very narrow agent of more limited utility (but easier toconstruct). Our present approach to populating the deductive space ispurely empirical, driven by human expert knowledge. The onus is thusupon the designer of the IA to generate a set of deductions, which(ideally) populate the space in a uniform manner. In reality, the set ofdeductions which inhabit the space can get become quite non-uniform(“clumpy”) given this empirical approach. Hence rigorous constraint onthe domain covered by an agent is entirely appropriate. Of course thisstrategy requires an appropriate mechanism at a higher abstract layer.However, the population of a higher layer agent can utilise the agentsbelow them in a behavioural manner thereby treating them as sub-spaces.

[0130] Once an agent's deductive space has been constructed andpopulated with deductions (points), it may be used to draw inferencesfrom observed facts. This is achieved by representing all available andrelevant facts as vectors in the multi-dimensional semantic space andconsidering how these vectors are located with respect to deductionpoints or volumes. A set of fact vectors, when added using vectoralgebra may precisely reach a deduction point in the space. In thatsituation, a deductive inference is implied. Alternatively, even in thesituation where no vectors or combinations of vectors precisely inhabitsa deduction point, more uncertain reasoning can be performed usingmechanisms such as distance metrics. For example, it may be implied thata vector, which is “close enough” to a deduction point, is a weakindicator of that deduction. Furthermore, in the face of partial data,vector techniques may be used to hone in on inferences by identifyingfacts (vectors), currently not asserted, which would allow for somesignificant deduction to be drawn. Such a situation may indicate thatthe system should perhaps direct extra resources towards discovering theexistence (or otherwise) of a key fact.

[0131] The actual inferencing mechanism to be used within higher-levelShapes Vector agents is slightly more flexible than the scheme we havedescribed above. Rather than simply tying facts to vectors defined interms of the IA's base nouns, we instead define an independent butspatially continuous ‘fact space’. FIG. 8 demonstrates the concept: adeductive space has been defined in terms of a set of base nounsrelevant to the IA. Occupying the same spatial region is a fact space,whose axes are derived from the agent's layer ontology. Facts aredefined as vectors in this second space: that is, they are entitiesfixed with respect to the fact axes. However, since the fact space anddeduction space overlap, these fact vectors also occupy a location withrespect to the base noun axes. It is this location which we use to makedeductive inferences based upon fact vectors. Thus, in the figure, theexistence of a fact vector (arrow) close to one of the deductions (dots)may allow for assertion of that deduction with a particular certaintyvalue (a function of exactly how close the vector is to the deductionpoint). Note that, since the axes of the fact space are independent ofthe axes of the deductive space, it is possible for the former to vary(shift, rotate and/or translate, perhaps independently) with respect tothe latter. If such a variation occurs, fact vectors (fixed with regardto the fact axes) will have different end-points in deduction-space.Therefore, after such a relative change in axes, a different set ofdeductions may be inferred with different confidence ratings. Thismechanism of semantic relativity may potentially be a powerful tool forperforming deductive inferencing in a dynamically changing environment.

[0132] An interesting aspect of our approach to vector-based deductiveinference is that it is based fundamentally upon ontological concepts,which can in turn be expressed as English nouns. This has the effectthat the deductions made by an agent will resemble simple sentences in avery small dialect of pseudo-English. This language may be a usefulmedium for a human to interact with the agent in a relatively naturalfashion.

[0133] While the inferencing strategy described above has someunorthodox elements in its approach to time-varying probabilisticreasoning for security applications, there are more conventional methodswhich may be used within Shapes Vector IA's in the instance that themethod falls short of its expected deductive potential.

[0134] As described above, the vector-based deductive engine is able tomake weak assertions of a deduction with an associated certainty value(based on distances in n-Dimensional space). This value can beinterpreted in a variety of ways to achieve different flavours ofdeductive logic. For example, the certainty value could potentially beinterpreted as a probability of the assertion holding true, derived froma consideration of the current context and encoded world knowledge. Suchan interpretation delivers a true probabilistic reasoning system.Alternatively, we could potentially consider a more rudimentaryinterpretation wherein we consider assertions with a certainty above aparticular threshold (e.g. 0.5) to be “possible” within a given context.Under these circumstances, the system would deliver a possiblistic formof reasoning. Numerous other interpretations are also possible.

[0135] Frame based systems offer one well understood (althoughinherently limited) alternative paradigm. Indeed, it is expected thatsome IA's will be frame based in any case (obtained off the shelf andequipped with an ontological interface to permit knowledge transfer withthe knowledge base).

[0136] Other agents based on neural nets, Bayesian, or statisticalprofiling may also inhabit the Agent macro-object.

[0137] 4.3 Other Applications

[0138] The IA architecture lends itself to other applications. Forexample, it is not uncommon for Defence organisations and institutionsto maintain many databases in just as many formats. It is very difficultfor analysts to peruse these databases in order to gain some requiredinsight. There has been much effort aimed at considering how particulardatabases may be structured in order for analysts to achieve theirobjectives. The problem has proved to be difficult. One of the majorhurdles is that extracting the analysts' needs and codifying them tostructure the data leads to different requirements not only betweenanalysts, but also different requirements depending on their currentfocus. One of the consequences is that in order to structure the datacorrectly, it must be context sensitive, which a relational database isnot equipped to handle.

[0139] Shapes Vector can overcome many of the extant difficulties bypermitting knowledge and deduction rules to be installed into an IA.This IA, equipped with a flexible user interface and strictly definedquery language, can then parse the data in a database in order to arriveat a conclusion. The knowledge rules and analyst-centric processing areencoded in the IA, not in the structure of the database itself, whichcan remain flat and context free. The Shapes Vector system allowsincremental adjustment of the IA without having to re-format andrestructure a database either through enhancement of the IA, or throughan additional IA with relevant domain knowledge. Either the IA makes theconclusion, or it can provide an analyst with a powerful tool to arriveat low level deductions that can be used to arrive at the desiredconclusion.

[0140] 5 Synthetic Stroboscopes and Selective Zoom

[0141] In this section, we discuss two mechanisms for overcomingdifficulties in bringing important events to the fore in a highlycluttered visual environment: Synthetic Strobes and Selective Zoom.

[0142] 5.1 Synthetic Strobes

[0143] One of the major difficulties with depicting data visually in areal-time system is determining how to handle broad temporal domains.Since the human is being used to provide inductive inference at themacro level, much data which needs to be represented visually may not bepossible to show due to temporal breadth. For example, there may be apattern in a fast packet stream, yet if we were to be able to see thepattern in the packet stream, other events which may also represent asignificant pattern may be happening much more slowly (e.g. slowlyrevolving sphere). Yet the perception of both patterns simultaneouslymay be necessary in order to make an inductive hypothesis.

[0144] A scientist at MIT during World War Two invented a solution tothis type of dilemma. By the use of a device (now well known in discosand dance studios) called a stroboscope, Edgerton was able to visualisepatterns taking place in one temporal domain in another. One of the moststriking and relatively recent examples was the visualisation ofindividual water droplets in an apparent stream produced by a rapidimpellor pump. The stream looked continuous, but viewed under thestrobe, each water droplet became distinctly apparent.

[0145] We can use the same concept of strobes, ie. synthetic strobes, tobring out multi temporal periodic behaviour in the Shapes Vectorvisualisation process. With a synthetic strobe, we can visualise packetflow behaviour more precisely, while still retaining a view of periodicbehaviour that may be occurring much more slowly elsewhere.

[0146] Since we have potentially many different events and objectswithin our view, it becomes necessary to extend the original strobeconcept so that many different types of strobes can be appliedsimultaneously. Unlike the employment of photonic based strobes, whichcan interfere with each other, we are able to implement strobes basedon:

[0147] Whole field of view

[0148] Per object instance

[0149] Per object class

[0150] Per object attribute

[0151] In addition, multiple strobes can be applied where each hascomplex periodic behaviour or special overrides depending on specificconditions. The latter can also be seen from the oscilloscopeperspective where a Cathode Ray Oscilloscope is triggered by an event inorder to capture the periodic behaviour. Naturally, with a syntheticstrobe, quite complex conditions can be specified as the trigger event.

[0152] Just as in the days of oscilloscopes, it is important to be ableto have variable control over the triggering rate of a strobe.Accordingly, control of the strobes is implemented via a set ofrheostats.

[0153] 5.2 Selective Zoom

[0154] In order to see a pattern, it is sometimes necessary to zoom outfrom a vista in order to gain a very high level view of activity in anetwork. While this can be quite useful, it is intuitive that importantevents for certain classes of object will fail to be noticed due to widedispersal across the vista. If a class of objects typically have a largeRepresentation compared to others, then zooming out to see a patternacross a large vista is appropriate. However, if the class of objects inquestion is small, then zooming out causes them to be less noticeablewhen compared to much larger objects.

[0155] Selective Zoom overcomes this difficulty and others of a similarilk by providing two mechanisms. The first mechanism allows a user tochange quickly the relative sizes of objects in relation to others. Thispermits a user to zoom out in order to see a large vista while stillretaining a discernible view of specific objects. The second mechanismpermits movement and projection of objects onto planes “above” or“below” the primary grids used to layout a view.

[0156] As can be seen in the following paragraphs, selective zoomprovides a generalised translation and rotation mechanism inthree-dimensional Cartesian space.

[0157] While the above two mechanisms can surely find utility, selectivezoom also provides a more sophisticated “winnowing” facility. Thisfacility caters to a typical phenomenon in the way humans “sift” throughdata sets until they arrive at a suitable subset for analysis. In thecase of focusing on a particular set of objects in order to undertakesome inductive or deductive analysis, a human may quickly select a broadclass of objects for initial analysis from the overall view despite apriori knowing that the selection may not be optimal. The user typicallythen undertakes either a refinement (selecting a further subset) orputting the data aside as a reference while reforming the selectioncriteria for selection. After applying the new criteria, the user maythen use the reference for refinement, intersection, or union withprevious criteria depending on what they see.

[0158] Via selective zoom (perhaps raised above the main view plane), auser can perform a selective zoom on a zoomed subset. This procedure canbe undertaken recursively, all the while making subsets from theprevious relative zoom. The effect can be made like a “staircasing” ofviews. FIG. 9 (segments two and three) depicts the use of selective zoomwhere subsets of nodes have been placed above the main view plane. Notethe set of nodes to the left were produced by a previous use of thezoom. This set need not be a subset of the current staircase.

[0159] Indeed the set to the left can be used to form rapidly a newselection criterion. The effects can be described by simple set theory.As implied above a user may also select any of the zoomed sets andtranslate them to another part of the field of view. These sets can alsothen be used again to form unions and intersections with other zoomedviews or subsets of views that are generated from the main view.

[0160] Segment one of FIG. 9 depicts the same view from above. Note theschematic style.

[0161] VDI has produced a visualisation toolkit in which a particularapplication depicts a set of machine nodes. By clicking on arepresentation of a node, it is “raised” from the map and so are thenodes to which it is connected. This may be interpreted as a simple formof one aspect of selective zoom. However, it is unclear whether this VDIapplication is capable of the range of features forming a generalisedselective zoom. For example, the capability to implement set translationin three dimensional Cartesian space, along with union and intersectionfor rapid reselection and manipulation of arbitrary view sets, as wellas relative size adjustment based on class, instance, or objectattribute properties.

[0162] 6 Temporal Hierarchies

[0163] Temporal hierarchies refer to three perceived issues: syntheticstrobes along both directions of the temporal axis; user informationoverload, and dealing with data streams with Intelligent Agents. Wediscuss each in turn.

[0164] 6.1 Strobes Revisited

[0165] In Section 5 we introduced the notion of a synthetic strobe whichcan be used to shunt rapid periodic behaviour along a “temporal axis” sothat the behaviour becomes discernible to the human eye. This shuntingwas necessary since many patterns of behaviour occur far too rapidly(e.g. characteristics of packet flow and their contents). However, alimitation of synthetic strobes as described is that they shunt or mappatterns in only one direction along the temporal axis. More precisely,rapid behaviour is shunted into a “slower” domain. Yet some behaviour ofsecurity significance may require a view which spans a relatively longtime. Hence it was hypothesised that strobes must be able to not onlyshow up rapid behaviour, but also show slow behaviour. To do this,Shapes Vector must be able to store events, and then be able to map astrobe over them in order to display the possible pattern. Essentially,it is preferable to be able to map behaviour, which can occur along abroad front of the temporal axis into a much smaller domain, which isperceptible to Humans. As an aside, it is a well known technique to seepatterns of motion in the cosmos by strobing and playing at high speedvarious observations, e.g. star field movement to ascertain thecelestial poles. However, what we propose here, apart from the relativenovelty of taking this concept into cyberspace, is the additionalunusual mechanism of complex trigger events in order to perceive the“small” events, which carry so much import over “long” time periods. Wecan assign triggers and functions on a scale not really envisaged evenin terms of cosmological playback mechanisms.

[0166] Elsewhere, we discuss many other issues related to syntheticstrobes. For example, the mechanisms for setting complex triggerconditions via “trigger boxes”, the need for “synthetic time”, itsrelation to real time, and generated strobe effects.

[0167] 6.2 User Information Overload

[0168] Another reason for using strobes, even if the pattern is alreadywithin the temporal perception domain of the user, is that they canhighlight potentially important behaviour from all the “clutter”.Visualisation itself is a mechanism whereby certain trends and macroevents can be perceived from an information rich data set. However, ifrelated or semantically similar events mix together, and a particularsmall event is to be correlated with another, then some form ofhighlighting is needed to distinguish it in the visual environment.Without this sort of mechanism, the user may suffer data overload.Synthetic strobes designed to trigger on specific events, and which onlyaffect particular classes of objects, are surmised to provide onemechanism to overcome this expected problem.

[0169] 6.3 Data Streams and IA's

[0170] One of the fundamental problems facing the use of IA's in theShapes Vector system is the changing status of propositions. Moreprecisely, under temporal shifts, all “facts” are predicates rather thanpropositions. This issue is further complicated when we consider thattypical implementations of IA's do not handle temporal data streams. Weaddress this problem by providing each IA with a “time aperture” overwhich it is currently processing. A user or a higher level agent can setthe value of this aperture.

[0171] Any output from an IA is only relevant to its time aperturesetting (FIG. 10). The aperture mechanism allows the avoidance of issuessuch as contradictions in facts over time, as well providing a finitedata set in what is really a data stream. In fact, the mechanism beingimplemented in our system permits multiple, non-intersecting aperturesto be defined for data input.

[0172] With time apertures, we can “stutter” or “sweep” along thetemporal domain in order to analyse long streams of data. Clearly, thereare a number of issues, which still must be dealt with. Chief amongstthese is the fact that an aperture may be set which does not, or ratherpartially, covers the data set whereby a critical deduction must bemade. Accordingly, strategies such as aperture change and multipleapertures along the temporal domain must be implemented in order toraise confidence that the relevant data is input in order to arrive atthe relevant deduction.

[0173] While we are aware that we can implement apertures in order tosupply us with useful deductions for a number of circumstances, it isstill an open question as to how to achieve a set of sweep strategiesfor a very broad class of deductions where confidence is high that weobtain what we are scanning for. One area, which comes to mind, is thenatural “tension” between desired aperture settings. For example, anaperture setting of 180 degrees (ie., the whole fact space) is desirableas this considers all data possible in the stream form the beginning ofthe epoch of capture to the end of time, or rather the last datacaptured. However, this setting is impractical from an implementationpoint of view, as well as introducing contradictions in the deductiveprocess. On the other hand, a very small aperture is desirable in thatimplementation is easy along with fast processing, but can result incritical packets not being included in the processing scan.

[0174] 7 Other Visualisation Efforts

[0175] Various techniques of visualisation have over the years beenapplied to the analysis of different domains of abstract data, withvarying success. Several such attempts bear similarities to portions ofthe Shapes Vector system, either in the techniques employed or the broadaims and philosophies guiding those techniques. In this section webriefly describe the most significant of these related visualisationefforts, concentrating on the specific domains of securityvisualisation, network visualisation and communications-related datamining.

[0176] The following discussion providing some background to theinvention is intended to facilitate a better understanding of theinvention. However, it should be appreciated that the discussion is notan acknowledgment or admission that any of the material referred to waspublished, known or part of the common general knowledge in any relevantcountry as at the priority date of the application.

[0177] 7.1 NetPARS

[0178] A proposal from NRaD and the NRL, the Network PropagationAssessment and Recovery System (NetPARS) is an effort to assist decisionmaking in defensive information warfare. It aims to supply such supportby means of rigorously tracking data quality within a system andestimating how degradations in quality propagate between data. Such aprotocol would, it is claimed, be capable of providing intrusiondetection services, assessment of security state and assist in recoveryfollowing an attack.

[0179] The proposed system architecture incorporates a set of mappingagents (responsible for keeping track of inter-relationships betweendata), sensor elements (capable of detecting intrusions and otherreductions in data quality) and recovery elements. When a sensor detectsthe compromise of one or more data item, the system computes (via aforward propagating expert system) the extent to which this loss inquality is propagated to other data. This information is presented tothe user to assist in the defence and/or containment of the compromise.

[0180] Ultimately it is envisaged that NetPARS will also incorporate asecond knowledge engine. This takes a reported reduction in data qualityand, by backward propagation, determines the tree of data items whichcould conceivably have been the initial cause of that reduction. Thisfault tree is a principal input to the process of recovery.

[0181] Although only sketchy details of the NetPARS proposal areavailable at present, the system would appear to have some superficialsimilarities to Shapes Vector. Both make use of forward and backwardpropagation of knowledge through a set of rules (although the functionof backward propagation is quite different in the two systems). Also,both NetPARS and Shapes Vector incorporate agents, which are tasked withintrusion detection as an aid towards a human response. However, whereasthe Shapes Vector architecture incorporates a broad range of suchagents, it seems that the intrusion detection functionality of NetPARSis currently limited to a single class of attack (storage spoofing).

[0182] Beyond these superficial resemblances the two systems have littlein common. NetPARS appears to place less importance upon visualisationtechnology, while in Shapes Vector this is an easily realisable featurewhere several novel visualisation techniques have been proposed. TheNRaD/NRL proposal appears to focus heavily on a tight domain of data andits inter-relationship, while the Shapes Vector system aims to model amuch larger concept space with a comprehensive ontology. Ontology can bemade relevant to a great variety of application areas. Computer securityas discussed in this specification is but one example. Shapes Vectoralso includes a potentially very powerful temporal control mechanism aswell as intelligent agent architecture with user semantic bindings.

[0183] 7.2 Security Visualisation

[0184] Eagle Netwatch is a commercial software package written by RaptorSystems Inc., which offers system administrators a visual representationof the security of their (firewall protected) network. The network isdisplayed by the tool as an interconnected set of coloured solidspositioned in a three-dimensional virtual world. By replaying audittrails collected on the firewall this display is animated to illustrateparticular gateway events which pertain to the system's security. Duringthe playback of this security “movie,” the user can rotate the virtualworld to more clearly observe the activities of particular networkelements. The tool also offers other visualisations of audit logs, mostnotably two-dimensional plots of gateway statistics against time.

[0185] The basic concept underlying Eagle Netwatch—that by observingevents in a visual representation of the network a (human user) maynotice patterns signifying security events—is similar to the ShapesVector philosophy as described in Section 3. However, at the time ofwriting this information the Netwatch tool lacks much of thesophistication of the Shapes Vector environment including the capacityfor real-time visualisation, the presence of intelligent deductiveagents, the possibility of remote discovery and visual mechanisms forrecognising temporal patterns.

[0186] 7.3 Network Visualisation

[0187] AT&T Bell have constructed a set of prototype tools, collectivelycalled SeeNet which provide tools for the visualisations oftelecommunications traffic. The system displays the traffic between twolocations by drawing a line on a two-dimensional geographical map. Linewidth and colour convey aspects of that traffic (e.g., volume). Invisualising traffic on an international scale, the resulting map istypically wrapped around a sphere to give the impression of the globe.By observing trends in the visualised traffic, key performancebottlenecks in real-world telecommunications services (including theInternet) have been identified. Also by investigating observed “hotspots” in these representations, AT&T have been able to identifyfraudulent use of their facilities.

[0188] A similar visualisation approach has been adopted by BritishTelecom in a prototype system for observing the parameters of theircommunications network. An outline map of Britain is overlaid with arepresentation of the BT network with a “skyscraper” projecting upwardsfrom each switching node. The height of the skyscraper denotes the valueof the metric being visualised (e.g., traffic or number of faults). Theuser can navigate freely through the resulting 3D environment. A secondvisualisation attempt undertaken by British Telecom considers adifferent three-dimensional visualisation of the communication networkas an aid for network architects. A similar approach has been adopted byIBM's Zurich Research Laboratories in their construction of a tool forvisualising a computer network's backbone within a fullthree-dimensional virtual (VRML) world. The goal of this latter systemis to ease the task of administering such network backbones.

[0189] While Shapes Vector can render similar scenes via its Geo Viewmethods, there is little else in common because of the existence of DataView, Selective View and Strobe when used as part of the visual element.The agent architecture and other elements further distinguish the ShapesVector system.

[0190] 7.4 Data Mining

[0191] The mining and visualisation of large data sets for the purposeof extracting semantic details is a technique that is applied to manyapplication domains. Several recent efforts have considered suchapproaches for deriving visual metrics for web-server performance andalso for conveying the inter-relatedness of a set of HTML documents.Research undertaken by the NCSA considers the first of these types ofdata mining in an immersive virtual reality environment called Avatar.The basic approach adopted in their performance measurement work is toconstruct a virtual world of “scattercubes”, regions of space in whichthree of the many measured metrics are plotted against one another. Theworld contains enough scattercubes that every set of three metrics iscompared in at least one. Users can browse this virtual world usingeither head-mounted displays or a virtual reality theatre, walkingwithin a single cube and flying over the whole aggregation of cubes.More recently this same system has also been used for visualising theperformance of massively parallel programs.

[0192] Other data-mining work has considered the derivation of semanticsrelated to the interconnections of WWW-based information. The WAVEenvironment from the University of Arkansas aims to provide a 3Dvisualisation of a set of documents grouped according to conceptualanalysis. Work at AT&T Bell considers plots of web-page access patternswhich group pages according to their place in a web site's documenthierarchy.

[0193] These efforts can be rendered with Shapes Vector's Data Viewdisplay. The Avtar effort does not, however, share the Shapes Vectorsystem's ability to effectively provide a semantic link between suchdata-oriented displays and geographic (or more abstract) views of theentities under consideration, nor represent the force paradigm'srepresented in Data View.

[0194] 7.5 Parentage and Autograph

[0195] Paretage and its successor Autograph are visualisation toolsconstructed by the NSA for assisting analysts in the task of locatingpatterns and trends in data relating to operating communicationsnetworks. The tools act as post-processors to the collected data,analysing the interactions between senders and receivers ofcommunications events. Based on this analysis the tools produce arepresentation of the network as a graph, with nodes describing thecommunications participants and the edges denoting properties of theaggregated communication observed between participants.

[0196] The user of the system may choose which of a pre-defined paletteof graph layouts should be used to render the graph to the screen. Thescalability of the provided layouts is limited and, as a means ofsupporting large data-sets, the tool allows for the grouping of nodesinto clusters which are represented as single nodes within the renderedgraph. Additionally, facilities exist for the displayed graph to beanimated to reflect temporal aspects of the collected data.

[0197] While the aims of the Parentage and Autograph systems have someintersection with the visual sub-systems of Shapes Vector, the systemsdiffer in a number of important regards. Firstly, the NSA software isnot designed for real-time analysis options. Secondly, the displaysgenerated by Parentage and Autograph are not intended to provide stronguser customisation facilities: the user may choose a layout from theprovided palette, but beyond this no control of the rendered graph isavailable. Contrast this with the Shapes Vector approach whichstipulates that each of the views of the security domain must beextremely customable to cater to the different abilities of users tolocate patterns in the visual field (see Section 3).

[0198] It is interesting to note that this last point has been observedin practical use of Parentage and Autograph: while the provided visualpalette allows some analysts to easily spot significant features, otherusers working with the same tools find it more difficult to locatenotable items.

Appendix Part 1—Custom Control Environments for Shapes Vector

[0199] As described in the body of this section of the specification,the Shapes Vector system is a tool based upon the fundamental assertionthat a user can visually absorb a large body of security-relevant data,and react. For such a capability and for a response to be effective, theShapes Vector user must have access to a broad range of hardwareperipherals, each offering a different style of interaction with thesystem. Section 2.2 of this part, describes the types of peripherals,which are present within the current system.

[0200] The exact physical configuration of peripherals presented to auser of the Shapes Vector system will depend upon the needs of the‘role’ that user is playing within the (collaborative) informationoperation. It is considered that there are two types of operationalroles: strategic/planning and tactical. Peripheral configurationscatering to the specific interactive needs of users operating in each ofthese modes are outlined below.

[0201] A.1 Strategic Environment

[0202] Since the principal functions of a strategic Shapes Vector userfocus primarily on non-real-time manipulation of data, there is littledemand for speedy forms of interaction such as that afforded byjoysticks and spaceballs. Instead, the core interactions availablewithin this environment must be extremely precise: we envisage the useof conventional modes such as keyboard entry of requests or commandscoupled with the gesture selection of items from menus (e.g. by mouse).Thus we would expect that a strategic Shapes Vector station mightconsist of a configuration similar to the traditional workstation: e.g.,a desk with screen, Keyboard and mouse atop.

[0203] A.2 Tactical Environment

[0204] In the course of a Shapes Vector information operation, one ormore of the operations team will be operating in a tactical mode. Insuch a mode, real-time data is being continually presented to the userand speedy (real-time) feedback to the system is of critical importance.Such interactions must primarily be made through high-bandwidthstream-based peripherals such as joysticks and dials. The complexity ofthe virtual environment presented by Shapes Vector suggests that a highnumber of different real-time interactions may be possible or desirable.

[0205] To provide a capacity for quickly switching between thesepossible functions, we choose to present the user with a large number ofperipherals, each of which is responsible for a single assignedinteraction. Since some system interactions are more naturallyrepresented by joysticks (e.g. flying through the virtual cyberspace)while others are more intuitively made using a dial (e.g. syntheticstrobe frequency) and so on, we must also provide a degree of variety inthe peripheral set offered to the user.

[0206] The technical issues involved in providing a large heterogeneousperipheral set in a traditional desktop environment are prohibitive. Tothis end a preferred design for a custom tactical control environmenthas been developed. The user environment depicted in FIG. 11 achievesthe goal of integrating a large number of disparate input peripheralsinto a dense configuration such that a user may very quickly shift andapply attention from one device to another.

[0207] The following input devices are incorporated into a preferredShapes Vector Tactical Control Station depicted in FIG. 11:

[0208] two joysticks

[0209] rudder pedals (not visible in the figure)

[0210] two dial/switch panels

[0211] keyboard (intended for the rare cases where slow but preciseinteraction is necessary)

[0212] trackball

[0213] The principal display for the tactical user is a large projectedscreen area located some distance in front of the control station.However, a small LCD screen is also provided for displaying localisedoutput (e.g. the commands typed on the keyboard).

PART 2 SHAPES VECTOR MASTER ARCHITECTURE

[0214] 1. Introduction

[0215] The fundamental aspects of the Intelligent Agent Architecture(IAA) for the Shapes Vector system are discussed in this Part of thespecification. Several unusual features of this architecture include ahierarchy of context free agents with no peer communication, a specificmethod for constructing ontologies which permits structured emergentbehaviour for agents fusing knowledge, and the ability to undertake asemantic inferencing mechanism which can be related to humaninterfacing.

[0216] 1.1 Shapes Vector Master Architecture

[0217] The master architecture diagram (FIG. 1) shows six mainsubsystems to Shapes Vector:

[0218] Sensor system. This sub-system comprises sensors that collectdata. A typical example would be an Ethernet packet sniffer. Sensors maybe local or remote and the communication path from the sensor and therest of the system can take many forms ranging from a UNIX socket,through to a wireless network data link.

[0219] The Intelligent Agent Architecture (Gestalt). This subsystem,described extensively in this paper, is responsible for processingsensor data and making intelligent deductions based on that input.

[0220] The Tardis. This sub-system is a real time manager of events anda global semantic mapper. It also houses the synthetic clock mechanismthat is discused in a later Part of this specification. The Tardis iscapable of taking deductions from the Agent Gestalt and mapping them toan event with a specific semantic ready for visualisation.

[0221] The Visuals. This sub-system actually comprises a number of“view” modules that can be regarded as sub-systems in their own right.Each view is built from common components, but visualises events inputto it from the Tardis according to a fundamental display paradigm. Forexample, Geoview displays events and objects based on a geographiclocation paradigm (wherein it is possible to layout objects according toa space coordinate system. Multiple interpretations of the layout arepossible. A typical use though is to layout computers and other physicalobjects according to their physical location.), whereas DataView laysout objects based on the level of interaction (forces) between them.

[0222] The I/O system. This sub-system provides extensive faculties forusers to navigate through the various views and interact with visualisedobjects.

[0223] The Configuration system. This sub-system offers extensivefeatures for customising the operation of all of the varioussub-systems.

[0224] Essentially, the system operates by recording data from thesensors, inputting it into the Agent Gestalt, where deductions are made,passing the results into the Tardis, which then schedules them fordisplay by the visualisation sub-system.

[0225] 1.2 Precis of this Part of the Specification.

[0226] Portions of the information contained in the following sectionswill be a repeat of earlier sections of the specification. This isnecessary due to the very large amount of information contained in thisdocument and the need to refresh the readers memory of the informationin the more detailed context of this part. Section 2 of this partdiscusses the fundamentals of the agent architecture, which includes adiscourse on the basic inferencing strategies for Shapes Vector agents.These inferencing strategies, described in Section 3 of this part arebased on epistemic principles for agents with a “low level ofabstraction” to a semantic vector based scheme for reasoning underuncertainty. Of interest is the method utilised to link an agent'ssemantics with the semantics of interaction with a user. This link isachieved by adjusting and formalising a highly restricted subset ofEnglish.

[0227] In Section 4 of this part the basic rules of constructing anagent are described and of how they must inhabit the architecturalframework. The architectural framework does not preclude theintroduction of “foreign” agents as long as an interface wrapper issupplied to permit it to transfer its knowledge and deduction via therelevant ontological interfaces.

[0228] Section 5 of this part discusses the temporal aspects ofintelligent agents. Section 6 of this part reveals some implications forthe development of higher abstraction levels for agents when consideringthe fusing of data from lower abstraction level agents. The ontologicalbasis for the first of these higher levels—levels 2 —are detailed inSection 7 of this part.

[0229] Section 8 of this part gives a brief overview of the requirementfor intelligent interfaces with which a user may interact with thevarious elements of an agent Gestalt. Section 9 of this part providessome general comments on the architecture, while Section 10 of this partcontrasts the system with the high-level work of Bass.

[0230] 2. The Agent Architecture

[0231] Shapes Vector is intended to house large numbers of IntelligentAgents (IA's), with different domains of discourse. These agents makeinferences and pass knowledge to one another in order to arrive at a setof deductions that permit a user to make higher level hypotheses.

[0232] 2.1 Agent Architecture

[0233] The Shapes Vector system makes use of a multi-layer multi-agentknowledge processing architecture. Rather than attempting to bridge theentire semantic gap between base facts and high-level security stateswith a single software entity, this gap is divided into a number ofabstraction layers. That is, we begin by considering the problem ofmapping between base facts and a marginally more abstract view of thenetwork. Once this (relatively easy) problem has been addressed, we moveon to considering another layer of deductive processing from thismarginally more abstract domain, to a yet more abstract domain.Eventually, within the upper strata of this layered architecture, thehigh-level concepts necessary to the visualisation of the network can bereasoned about in a straightforward and context-free fashion.

[0234] The resulting Shapes Vector Knowledge Architecture (SVKA) isdepicted in FIG. 7. The layered horizontal boxes within the figurerepresent the various layers of knowledge elements. At the very bottomof the figure lies the store of all observed base facts (represented asa shaded box). Above this lies a deductive layer (termed “Level 1” ofthe Knowledge Architecture) which provides the first level oftranslation from base fact to slightly more abstract concepts.

[0235] In order to achieve knowledge transfer between agents which isboth consistent and sound, an ontology (ie. a formal knowledgerepresentation) becomes imperative. Due to our approach of constructingour knowledge processing sub-system as a set of abstraction layers, wemust consider knowledge exchange at a number of different levels ofabstraction. To construct a single ontology capable of expressing allforms of knowledge present within the system is problematic due to thebreadth of abstraction. Attempting such ontology it is unlikely toproduce a tidy set of universal rules, and far more likely to produce acomplex family of inter-related concepts with ad-hoc exceptions. Morelikely, due to the total domain of discourse being so broad, ontologyproduced in this manner will be extremely context sensitive, leading tomany possibilities for introducing ambiguities and contradictions.

[0236] Taking a leaf from our earlier philosophy of simplificationthrough abstraction layering, we instead choose to define a set ofontologies: one per inter-layer boundary. FIG. 7 indicates theseontologies as curved arrows to the left of the agent stack.

[0237] The communication of factual knowledge to IAs in the first levelof abstraction is represented by means of a simple ontology of facts(called the Level 1 Shapes Vector Ontology). All agents described withinthis portion of the specification make use of this mechanism to receivetheir input. It is worthwhile noting that the knowledge domain definedby this ontology is quite rigidly limited to incorporate only a universeof facts—no higher-level concepts or meta-concepts are expressible inthis ontology. This simplified knowledge domain is uniform enough that areasonably clean set of ontological primitives can be conciselydescribed.

[0238] Interaction between IA's is strictly limited to avoid thepossibility of ambiguity. An agent may freely report outcomes to theShapes Vector Event Delivery sub-system, but inter-IA communication isonly possible between agents at adjacent layers in the architecture. Itis specifically prohibited for any agent to exchange knowledge with a“peer” (an agent within the same layer). If communication is to beprovided between peers, it must be via an intermediary in an upperlayer. The reasons underlying these rules of interaction are principallythat they remove chances for ambiguity by forcing consistentdomain-restricted universes of discourse (see below). Furthermore, suchrestrictions allow for optimised implementation of the KnowledgeArchitecture.

[0239] One specific optimisation made possible by theseconstraints—largely due to their capacity to avoid ambiguity andcontext—is that basic factual knowledge may be represented in terms oftraditional context-free relational calculus. This permits the use ofrelational database technology in storage and management of knowledge.Thus, for simple selection and filtering procedures on the knowledgebase we can utilise well known commercial mechanisms which have beenoptimised over a number years rather than having to build a customknowledge processor inside each intelligent agent.

[0240] Note that we are not suggesting that knowledge processing andretrieval is not required in an IA. Rather that by specifying certainrequirements in a relational calculus (SQL is a preferable language),the database engine assists by undertaking a filtering process whenpresenting a view for processing by the IA. Hence the IA can potentiallyreap considerable benefits by only having to process the (considerablysmaller) subset of the knowledge base which is relevant to the IA. Thisapproach becomes even more appealing when we consider that theimplementation of choice for Intelligent Agents is typically a logiclanguage such as Prolog. Such environments may incur significantprocessing delays due to the heavy stack based nature of processing onmodern Von Neumann architectures. However, by undertaking earlyfiltering processes using optimised relational engines and a simpleknowledge structure, we can minimise the total amount of data that isinput into potentially time consuming tree and stack-based computationalmodels.

[0241] The placement of intelligent agents within the various layers ofthe knowledge architecture is decided based upon the abstractionsembodied within the agent and the knowledge transforms provided by theagent. Two criteria are considered in determining whether a placement atlayer n is appropriate:

[0242] would the agent be context sensitive in the level n ontology? Ifso, it should be split into two or more agents. does the agent performdata fusion from one or more entities at level n? If so it must bepromoted to at least level n+1 (to adhere to the requirement of no“horizontal” interaction)

[0243] 2.2 A Note on the Tardis

[0244] A more detailed description of the Tardis is provided in part 5of the specification.

[0245] The Tardis connects the IA Gestalt to the real-time visualisationsystem. It also controls the system's notion of time in order to permitfacilities such as replay and visual or other analysis anywhere alongthe temporal axis from the earliest data still stored to the currentreal world time.

[0246] The Tardis is unusual in its ability to connect an arbitrarysemantic or deduction to a visual event. It does this by acting as avery large semantic patch-board. The basic premise is that for everyagreed global semantic (e.g. X window packet arrived [attribute list])there is a specific slot in an infinite sized table of globally agreedsemantics. For practical purposes, there are 2⁶⁴ slots and therefore thecurrent maximum number of agreed semantics available in our environment.No slot, once assigned a semantic, is ever reused for any othersemantic. Agents that arrive at a deduction, which matches the slotsemantic, simply queue an event into the slot. The visual system isprofiled to match visual events with slot numbers. Hence visual eventsare matched to semantics.

[0247] As for the well-known IP numbers and Ethernet addresses, theShapes Vector strategy is to have incremental assignment of semantics toslots. Various taxonomies etc. are being considered for slot grouping.As the years go by, it is expected that some slots will fall into disuseas the associated semantic is no longer relevant, while others areadded. It is considered highly preferable for obvious reasons, that noslot be reused.

[0248] As mentioned, further discussion about the Tardis and itsoperation can be found in part 5 of the specification.

[0249] 3. Inferencing Strategies

[0250] The fundamental inferencing strategy underlying Shapes Vector isto leave inductive inferencing as the province of the (human) user anddeductive inferencing as typically the province of the IA's. It isexpected that a user of the system will examine deductive inferencesgenerated by a set of IA's, coupled with visualisation, in order toarrive at an inductive hypothesis. This separation of duties markedlysimplifies the implementation strategies of the agents themselves.Nevertheless, we propose further aspects that may produce a verypowerful inferencing system.

[0251] 3.1 Traditional

[0252] Agents can employ either forward chaining or backward chaining,depending on the role they are required to fulfil. For example, someagents continuously comb their views of the knowledge base in attemptsto form current, up to date, deductions that are as “high level” aspossible. These agents employ forward chaining and typically inhabit thelower layers of the agent architecture. Forward chaining agents also mayhave data stream inputs from low level “sensors”. Based on these andother inputs, as well as a set of input priorities, these agents work togenerate warnings when certain security-significant deductions becometrue.

[0253] Another set of agents within the Shapes Vector system will bebackward chaining (goal driven) agents. These typically form part of the“User Avatar Set”: a collection of knowledge elements, which attempt toeither prove or disprove user queries (described more fully in Section 8of this part.).

[0254] 3.2 Possiblistic

[0255] In executing the possiblistic features incorporated into thelevel 2 ontology (described in Section 7.1 of this part), agents mayneed to resort to alternative logics. This is implied by the inherentmulti-valued nature of the possiblistic universe. Where a universe ofbasic facts can be described succinctly in terms of a fact existing ornot existing, the situation is more complex when symbolic possibility isadded. For our formulation we chose a three-valued possiblisticuniverse, in which a fact may be existent, non-existent, or possiblyexistent.

[0256] To reason in such a universe we adopt two different algebra's.The first a simple extension of the basic principle of unificationcommon to computational logic. Instead of the normal assignation ofsuccessful unifaction to existence and unsuccessful unification tonon-existence, we adopt the following:

[0257] successful unification implies existence,

[0258] the discovery of an explicit fact which precludes unificationimplies non-existence (this is referred to this as a hard fail),

[0259] unsuccessful unification without an explicit precluding caseimplies possible existence (this is referred to as a soft fail)

[0260] A second algebra, which may be used to reason in the possiblisticuniverse, involves a technique known as “predicate grounding” in which auser-directed pruning of a unification search allows for certainspecified predicates to be ignored (grounded) when possibilities arebeing evaluated.

[0261] 3.3 Vectors

[0262] Agents operating at higher levels of the Shapes Vector KnowledgeArchitecture may require facilities for reasoning about uncertain and/orincomplete information in a more continuous knowledge domain. Purelytraditional forward or backward chaining does not easily express suchreasoning, and the three-valued possiblistic logic may lack thenecessary quantitative features desired. To implement such agents analternative inferencing strategy is used based upon notions of vectoralgebra in a multi-dimensional semantic space. This alternative strategyis employed in conjunction with more conventional backward chainingtechniques. The use of each of the paradigms is dependent on the agent,and the domain of discourse.

[0263] Our vector-based approach to inferencing revolves aroundconstructing an abstract space in which relevant facts and deductionsmay be represented by geometrical analogues (such as points andvectors), with the proper algebraic relationships holding true. Ingeneral, the construction of such a space for a large knowledge domainis extremely difficult. For Shapes Vector, we adopt a simplifyingstrategy of constructing several distinct deductive spaces, each limitedto the (relatively small) domain of discourse of a single intelligentagent. The approach is empirical and is only feasible if each agent isrestricted to a very small domain of knowledge so that construction ofits space is not overly complex.

[0264] The definition of the deductive space for an IA is a methodicaland analytical process undertaken during the design of the agent itself.It involves a consideration of the set of semantic concepts (“nouns”)which are relevant to the agent, and across which the agent's deductionsoperate. Typically this concept set will contain elements of the agent'slayer ontology as well as nouns which are meaningful only within theagent itself. Once the agent's concept set has been discovered, we canidentify within it a subset of ‘base nouns’—concepts which cannot bedefined in terms of other members of the set. This identification isundertaken with reference to a semi-formal ‘connotation spectrum’ (acomparative metric for ontological concepts).

[0265] Such nouns have two important properties:

[0266] each is semantically orthogonal to every other base noun, and

[0267] every member of the concept set which is not a base noun can bedescribed as a combination of two or more base nouns.

[0268] Collectively, an IA's set of n base nouns defines a n-dimensionalsemantic space (in which each base noun describes an axis). Deductionsrelevant to the agent constitute points within this space; the volumebounded by spatial points for the full set of agent deductionsrepresents the sub-space of possible outputs from that agent. A rich setof broad-reaching deductions leads to a large volume of the space beingcovered by the agent, while a limited deduction set results in a verynarrow agent of more limited utility (but easier to construct). Ourpresent approach to populating the deductive space is purely empirical,driven by human expert knowledge. The onus is thus upon the designer ofthe IA to generate a set of deductions, which (ideally) populate thespace in a uniform manner.

[0269] In reality, the set of deductions that inhabit the space canbecome quite non-uniform (“clumpy”) given this empirical approach. Hencerigorous constraint on the domain covered by an agent is entirelyappropriate. Of course this strategy requires an appropriate mechanismat a higher abstract layer. However, the population of a higher layeragent can utilise the agents below them in a behavioural manner therebytreating them as sub-spaces.

[0270] Once an agent's deductive space has been constructed andpopulated with deductions (points), it may be used to draw inferencesfrom observed facts. This is achieved by representing all available andrelevant facts as vectors in the multi-dimensional semantic space andconsidering how these vectors are located with respect to deductionpoints or volumes. A set of fact vectors, when added using vectoralgebra may precisely reach a deduction point in the space. In thatsituation, a deductive inference is implied. Alternatively, even in thesituation where no vectors or combinations of vectors precisely inhabitsa deduction point, more uncertain reasoning can be performed usingmechanisms such as distance metrics. For example, it may be implied thata vector, which is “close enough” to a deduction point, is a weakindicator of that deduction. Furthermore, in the face of partial data,vector techniques may be used to hone in on inferences by identifyingFacts (vectors), currently not asserted, which would allow for somesignificant deduction to be drawn. Such a situation may indicate thatthe system should perhaps direct extra resources towards discovering theexistence (or otherwise) of a key fact.

[0271] The actual inferencing mechanism to be used within higher-levelShapes Vector agents is slightly more flexible than the scheme we havedescribed above. Rather than simply tying facts to vectors defined interms of the IA's base nouns, we can define an independent but spatiallycontinuous ‘fact space’. FIG. 8 demonstrates the concept: a deductivespace has been defined in terms of a set of base nouns relevant to theIA. Occupying the same spatial region is a fact space, whose axes arederived from the agent's layer ontology. Facts are defined as vectors inthis second space: that is, they are entities fixed with respect to thefact axes. However, since the fact space and deduction space overlap,these fact vectors also occupy a location with respect to the base nounaxes. It is this location which we use to make deductive inferencesbased upon fact vectors. Thus, in the Figure, the fact that the observedfact vector (arrow) is close to one of the deductions (dots) may allowfor assertion of that deduction with a particular certainty value (afunction of exactly how close the vector is to the deduction point).Note that, since the axes of the fact space are independent of the axesof the deductive space, it is possible for the former to vary (shift,rotate and/or translate, perhaps independently) with respect to thelatter. If such a variation occurs, fact vectors (fixed with regard tothe fact axes) will have different end-points in deduction-space.Therefore, after such a relative change in axes, a different set ofdeductions may be inferred with different confidence ratings. Thismechanism of semantic relativity may potentially be a powerful tool forperforming deductive inferencing in a dynamically changing environment.

[0272] An interesting aspect of the preferred approach to vector-baseddeductive inference is that it is based fundamentally upon ontologicalconcepts, which can in turn be expressed as English nouns. This has theeffect that the deductions made by an agent will resemble simplesentences in a very small dialect of pseudo-English. This language maybe a useful medium for a human to interact with the agent in arelatively natural fashion.

[0273] While the inferencing strategy described above has someunorthodox elements in its approach to time-varying probabilisticreasoning for security applications, there are more conventional methodsthat may be used within Shapes Vector IA's in the instance that themethod falls short of its expected deductive potential. Frame basedsystems offer one well understood (although inherently limited)alternative paradigm. Indeed, it is expected that some IA's will beframe based in any case (obtained off the shelf and equipped withontology to permit knowledge transfer with the knowledge base).

[0274] As described above, the vector-based deductive engine is able tomake weak assertions of a deduction with an associated certainty value(based on distances in n-Dimensional space). This value can beinterpreted in a variety of ways to achieve different flavours ofdeductive logic. For example, the certainty value could potentially beinterpreted as a probability of the assertion holding true, derived froma consideration of the current context and encoded world knowledge. Suchan interpretation delivers a true probabilistic reasoning system.Alternatively, we could potentially consider a more rudimentaryinterpretation wherein we consider assertions with a certainty above aparticular threshold (e.g. 0.5) to be “possible” within a given context.Under these circumstances, our system would deliver a possiblistic formof reasoning. Numerous other interpretations are also possible.

[0275] 3.4 Inferencing for Computer Security Applications

[0276] As presented, our IA architecture is appropriate to knowledgeprocessing in any number of domains. To place the work into theparticular context, for which it is primarily intended, we will nowconsider a simple computer security application of this architecture.

[0277] One common, but often difficult, task facing those charged withsecuring a computer network is detecting access of network assets whichappears authorised (e.g., the user has the proper passwords etc) but isactually malicious. Such access incorporates the so-called “insiderthreat” (i.e., an authorised user misusing their privileges) as well asthe situation where confidentiality of the identification system hasbeen compromised (e.g., passwords have been stolen). Typically,Intrusion Detection Systems are not good at detecting such securitybreaches, as they are purely based on observing signatures relating toimproper use or traffic.

[0278] Shapes Vector's comprehensive inferencing systems allow it todeduce a detailed semantic model of the network under consideration.This model coupled with a user's inductive reasoning skills, permitsdetection of such misuse even in the absence of any prior-known“signature”.

[0279] This application of Shapes Vector involves constructing a Gestaltof Intelligent Agents that are capable of reasoning about relativelylow-level facts derived from the network. Typically these facts would bein the form of observations of traffic flow on the network. Workingcollaboratively, the agents deduce the existence of computers on thenetwork and their intercommunication. Other agents also deduceattributes of the computers and details of their internal physical andlogical states. This information serves two purposes: one is to build upa knowledge base concerning the network, and another is to facilitatethe visualisation of the network. This latter output from the agents isused to construct a near real-time 3D visualisation showing thecomputers and network interfaces known to exist and theirinterconnection. Overlaid onto this “map” is animation denoting thetraffic observed by the agents, classified according to service type.

[0280] Observing such a Shapes Vector visualisation a user may note somevisual aspect that they consider being atypical. For example, the usermay note a stream of telnet packets (which itself might be quite normal)traversing the network between the primary network server and node whichthe visualisation shows as only a network interface. The implications ofsuch an observation are that a node on the network is generating aconsiderable body of data, but this data is formatted such that none ofthe Shapes Vector agents can deduce anything meaningful about thecomputer issuing the traffic (thus no computer shape is visualised, justa bare network interface).

[0281] The human user may consider this situation anomalous: given theirexperience of the network, most high volume traffic emitters areidentified quickly by one or more of the various IAs. While the telnetsession is legitimate, in as much as the proper passwords have beenprovided, the situation bears further investigation.

[0282] To probe deeper, the User Avatar component of Shapes Vector,described more fully in Section 8 in Part 2 of the specification, can beused to directly query the detailed knowledge base the agents have builtup behind to the (less-detailed) visualisation. The interaction in thissituation might be as follows:

[0283] human>answer what User is-logged-into Computer “MainServer”?

[0284] gestalt>Relationship is-logged-into [User Boris, ComputerMainServer]

[0285] This reveals a user name for the individual currently logged intothe server. A further interaction might be:

[0286] human>find all User where id=“Boris”?

[0287] gestalt>Entity User (id=Boris, name=“Boris Wolfgang”, type=“guestuser”)

[0288] An agent has deduced at some stage of knowledge processing thatthe user called Boris is logged in using a guest user account. TheShapes Vector user would be aware that this is also suspicious, perhapseliciting a further question:

[0289] human>answer what is-owned-by User Boris”? gestalt> Relationshipis-owned-by [File passwords, User Boris]   Relationship is-owned-by[Process keylogger, User Boris]   Relationship is-owned-by [ProcesspasswordCracker, User Boris]

[0290] The facts have, again, been deduced by one or more of the IA'sduring their processing of the original network facts. The human user,again using their own knowledge and inductive faculties, would becomemore suspicious. Their level of suspicion might be such that they takeaction to terminate Boris' connection to the main server.

[0291] In addition to this, the user could ask a range of possiblisticand probabilistic questions about the state of the network, invokingfaculties in the agent Gestalt for more speculative reasoning.

[0292] 3.4 Other Applications

[0293] The IA architecture disclosed herein lends itself to otherapplications. For example, it is not uncommon for the Defence communityto have many databases in just as many formats. It is very difficult foranalysts to peruse these databases in order to gain useful insight.There has been much effort aimed at considering how particular databasesmay be structured in order for analysts to achieve their objectives. Theproblem has proved to be difficult. One of the major hurdles is thatextracting the analysts' needs and codifying them to structure the dataleads to different requirements not only between analysts, but alsodifferent requirements depending on their current focus. One of theconsequences is that in order to structure the data correctly, it mustbe context sensitive, which a relational database is not equipped tohandle.

[0294] Shapes Vector can overcome many of the extant difficulties bypermitting knowledge and deduction rules to be installed into an IA.This IA, equipped with a flexible user interface and strictly definedquery language, can then parse the data in a database in order to arriveat a conclusion. The knowledge rules and analyst-centric processing areencoded in the IA, not in the structure of the database itself, whichcan thus remain context free. The Shapes Vector system allowsincremental adjustment of the IA without having to re-format andrestructure a database through enhancement of the IA, or through anadditional IA with relevant domain knowledge. Either the IA makes theconclusion, or it can provide an analyst with a powerful tool to arriveat low level deductions that can be used to arrive at the desiredconclusion.

[0295] 4. Rules for Constructing an Agent

[0296] In Section 2 of this part of the specification, several rulesgoverning agents were mentioned, e.g. no intra level communication andeach agent must be context free within its domain of discourse.Nevertheless, there are still a number of issues, which needclarification to see how an agent can be constructed, and some of theresultant implications.

[0297] In a preferred arrangement the three fundamental rules thatgovern the construction of an agent are:

[0298] 1. All agents within themselves must be context free;

[0299] 2. If a context sensitive rule or deduction becomes apparent,then the agent must be split into two or more agents;

[0300] 3. No agent can communicate with its peers in the same level. Ifan agent's deduction requires input from a peer, then the agent must bepromoted to a higher level, or a higher level agent constructed whichutilises the agent and the necessary peer(s).

[0301] In our current implementation of Shapes Vector, agentscommunicate with other entities via the traditional UNIX socketsmechanism as an instantiation of a component control interface. Theagent architecture does not preclude the use of third party agents orsystems. The typical approach to dealing with third party systems is toprovide a “wrapper” which permits communication between the system andShapes Vector. This wrapper needs to be placed carefully within theagent hierarchy so that interaction with the third party system ismeaningful in terms of the Shapes Vector ontologies, as well aspermitting the wrapper to act as a bridge between the third party systemand other Shapes Vector agents. The wrapper appears as just another SVagent.

[0302] One of the main implications of the wrapper system is that it maynot be possible to gain access to all of the features of a third partysystem. If the knowledge cannot be carried by the ontologies accessibleto the wrapper, then the knowledge elements cannot be transportedthroughout the system. There are several responses to such cases:

[0303] 1. The wrapper may be placed at the wrong level.

[0304] 2. The Ontology may be deficient and in need of revision.

[0305] 3. The feature of the third party system may be irrelevant andtherefore no adjustments are required.

[0306] 5. Agents and Time

[0307] In this section we discuss the relationship between the operationof agents and time. The two main areas disclosed are how the logic basedimplementation of agents can handle data streams without resorting to anembedded, sophisticated temporal logic, and the notion of synthetic timein order to permit simulation, and analysis of data from multiple timeperiods.

[0308] 5.1 Data Streams and IA's

[0309] One of the fundamental problems facing the use of IA's in theShapes Vector system is the changing status of propositions. Moreprecisely, under temporal shifts, all “facts” are predicates rather thanpropositions. This issue is further complicated when we consider thattypical implementations of an IA do not handle temporal data streams.

[0310] We address this problem by providing each IA with a “timeaperture” over which it is currently processing. A user or a higherlevel agent can set the value of this aperture.

[0311] Any output from an IA is only relevant to its time aperturesetting (FIG. 10). The aperture mechanism allows the avoidance of issuessuch as contradictions in facts over time, as well providing a finitedata set in what is really a data stream. In fact, the mechanism beingimplemented in our system permits multiple, non-intersecting aperturesto be defined for data input.

[0312] With time apertures, we can “stutter” or “sweep” along thetemporal domain in order to analyse long streams of data. Clearly, thereare a number of issues, which still must be addressed. Chief amongstthese is the fact that an aperture may be set which does not, or ratherpartially, covers the data set whereby a critical deduction must bemade. Accordingly, strategies such as aperture change and multipleapertures along the temporal domain must be implemented in order toraise confidence that the relevant data is input in order to arrive atthe relevant deduction.

[0313] While we are aware that we can implement apertures in order tosupply us with useful deductions for a number of circumstances, it isstill an open question on how to achieve an optimal set of sweepstrategies for a very broad class of deductions where confidence is highthat we obtain what we are scanning for. One area, which comes to mind,is the natural “tension” between desired aperture settings. For example,an aperture setting of 180 degrees (ie., the whole fact space) isdesirable as this considers all data possible in the stream from thebeginning of the epoch of capture to the end of time, or rather the lastdata captured. However, this setting is impractical from animplementation point of view, as well as introducing potentialcontradictions in the deductive process. On the other hand, a very smallaperture is desirable in that implementation is easy along with fastprocessing, but can result in critical packets not being included in theprocessing scan.

[0314] Initial test of an agent, which understands portions of the HTTPprotocol, has yielded anecdotal evidence that there may be optimumaperture settings for specific domains of discourse. HTTP protocol datafrom a large (5GB) corpus were analysed for a large network. It wasshown that an aperture setting of 64 packets produced the largest set ofdeductions for the smallest aperture setting while avoiding theintroduction of contradictions.

[0315] The optimal aperture setting is of course affected by the datainput, as well as the domain of discourse. However, if we determine thatour corpus is representative of expected traffic, then default optimalaperture setting is possible for an agent. This aperture setting needonly then be adjusted as required in the presence of contradictingdeductions or for special processing purposes.

[0316] 5.2 Temporal Event Mapping for Agents

[0317] In the previous section, we discussed how an agent could havetime apertures in order to process data streams. The issue of time isquite important, especially when considering that it takes a finiteamount of time for a set of agents to arrive at a deduction and presenta visualisation. Also, a user may wish to replay events at differentspeeds in order to see security relevant patterns. To provide suchfacilities in Shapes Vector, we introduce the notion of a syntheticclock. All entities in the system get their current time from thesynthetic clock rather than the real system clock. A synthetic clock canbe set arbitrarily to any of the current or past time, and its rate ofchange can also be specified.

[0318] A synthetic clock allows a user to run the system at differentspeeds and set its notion of time for analysing data. The syntheticclock also permits a variety of simulations to be performed under anumber of semantic assumptions (see Section 7 of this part of thespecification)

[0319] The above is all very well, but Shapes Vector may at the sametime be utilised for current real-time network monitoring as well asrunning a simulation. In addition, the user may be interested incorrelating past analysis conditions with current events and vice versa.For example, given a hypothesis from an ongoing analysis, the user maywish to specify that if a set of events occur in specific real-timewindows based on past event temporal attributes or as part of an ongoingsimulation, then an alarm should be given and the results or specificattributes can flow bi-directionally between the past event analysis andthe current event condition. Hence Shapes Vector should be able tosupply multiple synthetic clocks and the agent instances runningaccording to each clock must be distinguishable from each other. Allsynthetic clocks are contained in the Tardis that is discussed in detailin Part 5 of this specification.

[0320] 6. Implications for Higher Level Agents

[0321] The criterion that all agents must be context free is in fact,not fully achievable. There are a number of influencing factors, butchief amongst these is time. An agent judged to be context free oneyear, may not be context free later in its lifecycle, despite no changeto its content. For example, consider a simple agent responsible foranalysing the headers of web traffic (HTTP) to determine which requestswent via a proxy server. At the time such an agent is written it may becontext free (or more precisely it's context is the universally acceptedrules of HTTP transactions). However, future changes to the HTTPprotocol or to the common practices used by web browsers or servers maycause it to become context sensitive despite no changes to the agentitself. That is, all deductions produced by the agent become true, onlyin the context of “how HTTP worked at the time the agent was written”.

[0322] The above tends to encourage all agents to hold only one simpleor “atom” deduction. This then ensures context freedom over a very longperiod of time. However, there are at least a couple of practicaldifficulties to such an approach:

[0323] 1. A definition of what constitutes an atom deduction that isvalid for all of the architecture must be determined;

[0324] 2. A very sophisticated criterion for placement of agents withinthe agent hierarchy is needed to the extent that a complete metalogic ofsemantics right across the agent architecture would be needed(practically impossible).

[0325] 7. Higher Level Ontologies

[0326] Detail of how the ontologies contribute to the functioning of theAgent architecture is disclosed in this section. In particular, there isfocus on the ontologies above level 1, and provision of a briefdiscourse of the two lowest levels.

[0327] 7.1 Level 2

[0328] In developing the level 2 ontology, it became apparent thatattempting the same approach as for level 1 would not work. Level 1focuses very much on “concrete” objects (e.g. modems, computer) anddeterministic concrete relationships (e.g. connection) in the form of atraditional first order logic. Adopting a similar approach for level 2proved difficult in the light of the desirable criteria for higher levelontologies, namely that they should:

[0329] Seek to embody a higher level of abstraction (relative to theprevious, lower, level ontologies).

[0330] Seek description in terms of “atomic” relationships for eachabstraction level, from which more complex relationships can be built.

[0331] Offer opportunities for fusion activities, which cannot behandled at, lower layers (since they would be context sensitive).

[0332] Given the above criteria, the identification of a set oforthogonal, higher-level object types or classes on which to base acontext-free level 2 ontology was problematic. A more promisingconstructive methodology for level 2 was to focus less on objects in andof themselves (as the level 1 ontology had done) and instead to identifya set of fundamental operations and relationships. That is, to movetowards a description in terms of higher-level logics.

[0333] The chosen approach for constructing the level 2 ontology was toconsider the types of knowledge-based relations and operators an agentoperating at level 2 would require to support Shapes Vector's securitymission. Such agents would necessarily need to conduct semanticmanipulations of basic objects and concepts embodied in level 1.Operators that remain generic (like those in level 1) were preferredover security-specific semantics. The key operators and relationspresent within the ontology are:

[0334] 7.1.1 Relationships

[0335] These relationships may appear in both ontological statements(assertions) and also as clauses in ontological queries.

[0336] Simple Set Theoretic Operators. A suite of common set-basedrelationships are incorporated, including set membership (Member_of),set disjunction (Intersection_of), set conjunction (Union_of), andCartesian_product_of. These relationships provide the traditional basisfor constructing more complex semantic relationships. Using suchrelationships we can, for example, express that computer“dialup.foo.net.au” is a member of the set of computers that have sentsuspicious mail messages to server “www.bar.com” in the past day.

[0337] Consistency Operators: Consistent_with, Inconsistent_with. Theuse of these relationships takes the form “X consistent_with Y” or “Xinconsistent_with Y”. Since we are at a higher level, it is clear thatcontradictions will become apparent which are either invisible to thelower level agents, or as a result of their aperture settings, caused bya temporal context sensitivity. For example, we can use the operator toexpress the fact that a conclusion made by an e-mail agent that ane-mail originated at “nospam.com” is inconsistent with anotherobservation made by a different agent that web traffic from the samemachine reports its name as “dialup.foo.net.au”.

[0338] It is important to distinguish between this relationship and thetraditional logical implies. We cannot construct a practicalimplementation of implies in our system. There are several well-knowndifficulties such as an implementation of a safe form of the “not”operator. Hence we have avoided the issue by providing a more restrictedoperator with a specific semantic which nevertheless serves our purposesfor a class of problems.

[0339] Based_on. The above Consistent_with and Inconsistent_withrelationships are not sufficient for expressing practical semantics ofconsistency in Shapes Vector. Given the broad ranging domains of lowerlevel agents, these relationships beg the question “consistent (orinconsistent) under what basis?”. Hence the Based_on clause which isused in the following manner “X Consistent_with Y Based_on Z”. The rulesof such a logic may be derived from human expert knowledge, or may beautomatically generated by a computational technique able to drawconsistency relationships from a corpus of data. Here, Z representsconsistency logic relevant to the particular context. An implication isthat a simple form of type matching is advisable in order to preventuseless consistency logics being applied to elements being matched forconsistency. The type matching can be constructed by utilising the settheory operators.

[0340] Predicated Existential Operator: Is_Sufficient_for. Thisrelationship takes the form “X is_sufficient_for Y” and encapsulates thesemantics that Y would be true if X could be established. That is, it isused to introduce a conditional assertion of X predicated on Y. Thisfacility could be used, for example, to report that it would beconclusively shown that computer “dialup.foo.net.au” was a web server IFit were observed that “dialup.foo.net.au” had sent HTML traffic on port80.

[0341] Possiblistic Existential Operator: Possible (X). Thisrelationship serves to denote that the fact contained within itsparentheses has been deemed to be a definite possibility.

[0342] That is, the generator of the statement has stated that while itmay not be able to conclusively deduce the existence of the fact, it hasbeen able to identify it as a possibility. This relationship isnecessary in order to be able to handle negation and the various formsof possibility. Further discussion appears below. Any fact expressiblein the level 1 or level 2 ontology may be placed within a Possiblestatement. The most typical use of this operator would be in a responseto a possiblistic query (see below). Note that the possibility relationdoes not appear as an operator in a query.

[0343] 7.1 .2 Interrogative Operators

[0344] The above relationships (except for Possible) also appear asoperators in queries made to the Agent Gestalt at level two. However,there are a number of operators, which do not have a correspondingrelation in the ontology. These are now discussed:

[0345] the usual boolean operators which can also be expressed in termsof set theory are supplied.

[0346] asserting (Y). This unary operator allows us to ask whether theproposition X is true if we assume Y is a given (ie. whether X can beestablished through the temporary injection of fact Y into theuniverse). Y may or may not be relevant in deciding the truth of X hencethe operator is in stark contrast to the Is_Sufficient_For relationwhere the truth of Y directly implies the truth of X. There are someinteresting complexities to implementing the asserting operator in aProlog environment. The assertion must take place in a manner such thatit can override a contrary fact. For most implementations, this meansensuring that it is at the head of any identical clauses. One of theimplementation methods is to first work out whether Y is true in thefirst place and if not, place in a term with a different arity anddirect the query to the new term in order to bypass the other searchpaths.

[0347] Is_it_possible. This operator allows for possiblistic queries. Ittakes the form “Is_it_possible X” where X may be any level 1 or level 2ontological construct. Specifically, ontological relationships may beused, e.g., “Is_it_possible X [relationship (e.g. Member_of)] Y”.Is_it_possible can be used in conjunction with the asserting operator(e.g. Is_it_possible X [operator] Y asserting Z) to perform apossiblistic query where one or more facts are temporarily injected intothe universe. Using this operator we can, for example, issue a queryasking whether it is possible that computer “dialup.foobar.net.au” is aweb server. Furthermore we could ask whether based on an assumption thatcomputer “dialup.foo.net.au” is connected via a modem, the computermakes use of a web proxy. Is_it_possible provides a means for returningresults from predicates as ground facts rather than insisting that allqueries resolve to an evaluated proposition (see Section 3.2 of thispart). The evaluation result of a query of this nature will returneither no, or maybe. The maybe result occurs if it is possible or thereis no condition found which bars the condition, or no if a condition canbe found in the universe preventing its possibility.

[0348] Is_it_definitely_possible. This operator is not orthogonal to theprevious one. The evaluation result is either yes, or no. The differencebetween this operator and the previous one is that for it to returntrue, there must be a set of conditions in the universe which permit theresult to be true, and the relation possibility exists.

[0349] Under_what_circumstances. This operator provides for a reversestyle of possiblistic querying in which a target fact is given and thequeried entity is called upon to provide the list of all conditions thatwould need to hold true for that fact to be established. For example wecan ask under what conditions would it be conclusively true that a guestuser had remotely logged into the computer “dialup.foobar.net.au”.

[0350] Not is one of the more interesting operators. There has been muchdiscussion over the years on how to implement the equivalent of logicalnegation. The problems in doing so are classic and no general solutionis disclosed. Rather, three strategies are generated that provide animplementation approach, which satisfies our requirement for a logicalnegation operator. For the Shapes Vector system, any Not operator istransferred into a possiblistic query utilising negation as failure. Not(x) is transformed to the negation of Is_it_possible (X). This is wherenegation operation maps ‘no’ to ‘yes’, and maps ‘maybe’ to ‘maybe’.Doing so requires us to have the user make an interpretation of theresult based on fixed criteria. However, it is claimed that such aninterpretation is simple. For example: a user may inquire as to whetherit is “not true that X is connected to Y”. This would be transformedinto a query as to whether it was possible that X is connected to Y andthe result of that second query negated. If the system determined thatit might be possible that X is connected to Y, the final response wouldbe that it might be possible that it is “not true that X is connected toY.” Alternatively, if it could be established that the connection wasnot possible, the final response would be yes it is “not true that X isconnected to Y.”

[0351] The above possibility operators cause some interestingimplementation issues. It needs to be possible to detect the reason whya query fails, ie. did it fail due to a condition contradicting success(hard fail), or that simply all goals are exhausted in trying to find amatch (soft fail). As a partial solution to this issue, we must add tothe criteria for constructing an agent. A further criterion is that anagent's clauses are constructed in two sets: case for the positive, andcase for the negative. We attempt to state explicitly the negativeaspects. These negative clauses, if unified, cause a hard fail to beregistered. It is fairly simple to deduce here that we cannot guaranteecompleteness of an agent across its domain of discourse. However, a softfail interpretation due to incompleteness of the part of the agentremains semantically consistent with the logic and the response to theuser.

[0352] 7.2 Level 3 and Above

[0353] As can be seen, the main characteristics of level 2 when comparedto level 1 are the inclusion of possibilistic reasoning and theintroduction of the ability to define semantics for consistency. If wecarry this abstraction path (ie. first order logic to possibilisticlogic) one step further we can surmise that the next fundamental stepshould be an ontology which deals with probabilistic logic. For example,semantics to support operators such as “likely”.

[0354] Initial operators designated for level three include “is itlikely” which has a set of qualifiers in order to define what “likely”means. Interpretation based on specific user profiles will be neededhence user Avatars (see next section in this portion of thespecification) are present in order to help interpret abstract userqueries into precise, complex ontology. It is suggested that any levelsbeyond this become much more mission specific and will begin to includesecurity specific relationships.

[0355] In actual fact, the labelled levels “2” and “3” may not beactually be located consecutively at the second and third layers of theagent hierarchy. Due to the need for avoiding context sensitivity withina level when introducing new agents, there will always be a need tointroduce intermediate levels in order to cater for fusers in a way thatdoes not necessitate the expansion of the adjacent levels' ontologies.Hence we refer here to the labels “level 2” and “level 3” as ontologicaldelineators. Indeed, current expectations are that the possibilisticreasoning parts of an ontology will be introduced around level six dueto fusing agents which are to be introduced for Shapes Vector's securitymission.

[0356] 7.3 An Example of Possiblistic Querying

[0357] Consider a simple IA/fuser designed to accept input from twoknowledge sources—one describing network hosts and the ports they listenon, and another describing local file system accesses on a network host.By fusing such inputs, the agent deduces situations where security hasbeen compromised.

[0358] Such an agent may contain the following rules (specified here inpseudo-English): In reality, the deductive rules within such an agentwould be considerably more complex and would involve many additionalfactors. The rules are simplified here for illustrative purposes.

[0359] 1. If a process Y listens on port P AND P is NOT a recognisedport<1024 THEN Y is a “non-system daemon”.

[0360] 2. If a process Y is a non-system daemon AND Y wrote to systemfile F THEN Y “corrupted” F.

[0361] Consider the situation where, in analysing the data for a timewindow, the agent receives the following input:

[0362] Process 1234 listens on port 21

[0363] Process 3257 has written to !etc/passwd

[0364] Process 1234 has written to /etc!passwd

[0365] Process 3257 listens on port 31337

[0366] Process 987 listens on port 1022

[0367] Port 21 is a recognised port

[0368] The following possiblistic queries may be issued to the agent:

[0369] Is it possible Process 1234 corrupted /etc!passwd?

[0370] In this case, the agent would generate a Hard Fail (i.e. a“definite no”) since a contradiction is encountered. The relationship“corrupted” can only be true if Rule 1 has classified Process 1234 as a“non-system daemon”, but that can only happen if Port 21 is not“recognised”. This last fact is explicitly contradicted by the availablefacts.

[0371] Is_it_possible Process 987 corrupted /etc!passwd?

[0372] In this case, the agent would generate a Soft Fail (i.e., a“maybe”) since, while no contradiction is present, neither is theresufficient evidence to conclusively show Process 987 has corrupted/etc/passwd. Rule 1 can classify Process 987 as a non-system daemon, butthere are no observations showing that Process 987 wrote to /etc/passwd(which does not, in itself mean that it did not, given the agent'sinherently incomplete view of the world).

[0373] Under_what_circumstances could Process 987 have corrupted/etc/passwd?

[0374] In this case the agent would respond with the fact “Process 987has written to /etc/passwd”, which is the missing fact required to showthat the process corrupted /etc/passwd.

[0375] Is it possible Process 3257 corrupted /etc/passwd?

[0376] Not only is it possible that Process 3257 could have corruptedthe file, there is sufficient evidence to show that it definitelyoccurred. That is, under normal predicate logic the rules would deducethe “corrupted” relationship. However, since the Is_It_Possible operatorreplies either “no or “maybe”, the agent in this case replies “maybe”.

[0377] Can you show that Process 3257 corrupted /etc/passwd?

[0378] This is a straight predicate (i.e., non-possiblistic) query.Since the facts support a successful resolution under the Rules 1 and 2,the agent replies “yes”.

[0379] 7.4 An Example of the Use of Consistency

[0380] In this section, we describe a simple example showing the utilityof the consistency logic for a security application.

[0381] Consider the case of a simple consistency agent, whichunderstands the basics of the TCP protocol, and in particular is awareof the traditional “three-way handshake” involved in the establishmentof a TCP connection. This agent would be able to recognise validhandshakes and report the consistency of the packet sequences theycomprise. Consider the following input to such an agent:

[0382] Packet L1 (type=“TCP SYN”)

[0383] Packet L2 (type=“TCP SYN ACK”)

[0384] L2 directly-follows L1

[0385] Packet L3 (Type=“TCP ACK”)

[0386] L3 directly-follows L2

[0387] For this input, the agent will recognise the validity of thishandshake and be able to report consistency of the packet sequences bystating:

[0388] (L2 directly-follows L1) Consistent_with (L3 directly-follows L2)Based_on “TCP Handshake (Packet L1 (type=\“TCP SYN\”), Packet L2(type=\“TCP SYN ACK\”), Packet L3 (Type=\“TCP ACK\”))”

[0389] Alternatively, the same agent could be presented with an invalidhandshake as input, for example:

[0390] Packet X1 (type=TCP SYN”)

[0391] Packet X2 (type=“TCP SYN ACK”)

[0392] X2 directly-follows X1

[0393] Packet X3 (Type=“TCP RST”)

[0394] X3 directly-follows X2

[0395] In this case the agent would recognise that it is invalid for aTCP implementation to complete two parts of the handshake and thenspontaneously issue a Reset packet 0. It would represent thisinconsistency by reporting: (X2 directly-follows X1) Inconsistent_with(X3 directly-follows X2) Based_on “TCP Handshake (Packet X1 (type=\“TCPSYN\”), Packet X2 (type=\“TCP SYN ACK\”), Packet X3 (Type=\“TCP RST\”))”

[0396] Such a statement of inconsistency may be directly interrogated bya user interested in anomalous traffic, or alternatively passed as inputto a set of security-specific agents, which would correlate theobservation with other input.

[0397] An interesting implementation issue arises when we consider theconstruction of consistency assertions. The number of assertions todescribe consistency, e.g. for TCP/IP traffic, may be very large, ordependent on specified environments and it could be data driven. Thereis a surprisingly simple possibility for the automatic generation ofconsistency assertion sets. Very preliminary investigation has indicatedthat data mining methods on designated standard data corpus are verysuited for generating assertion sets, which may then be used as theconsistency logic. Data mining is extensively used in detectingvariances in traffic, but has been less successful in detectingintrusions. However, data mining has shown to be very successful incharacterising data, and thus is proving an exciting possibility for usein the Shapes Vector system for describing bases of consistency.

[0398] 8. User Avatars

[0399] It is necessary to have an intelligent interface so that the usermay interact with the agents as a Gestalt. Accordingly, a set of useravatars is constructed. These avatars preferably contain a level ofintelligent processing as well as the usual query parsing as a result ofin one example, commercial voice recognition packages. In order tomaintain consistency, user avatars are apparent at all levels in theontologies. This permits each avatar to be able to converse with theagents at its level, while still permitting control and communicationmethods with avatars above and below. Put simply, the same reasons fordeveloping the agent hierarchy are applied to the avatar set. Given thenature of an avatar, it may be argued by some that there is littledifference between an agent in Gestalt, and the avatar itself. Avatarsand Gestalt agents are distinguished by the following characteristics:

[0400] Agents deal with other agents and Avatars.

[0401] Avatars deal with agents and users.

[0402] Avatars can translate user queries into precise ontology based onspecific user driven adaptive processes to resolve context.

[0403] Further to the above, Avatars store user profiles in a manner soas to interpret different connotations based on specific useridiosyncrasies. For example, the use of the probabilistic logic basedqueries where the term likely can be weighted differently according toeach user.

[0404] One of the activities expected of Avatars in the Shapes Vectorsystem is to modify queries so that they may be made more precise beforepresentation to the Gestalt. For example, at a high layer ofabstraction, a user may initiate the query “I have observed X and Y, amI being attacked?”. An Avatar, given a user profile, may modify thisquery to “Given observations X, Y, based on Z, is it likely that a knownattack path exists within this statistical profile”.

[0405] 9. Further Comments on the Architecture

[0406] The hierarchical layering of the architecture with interleavedontologies provides a strong advantage to Shapes Vector. Each ontologyprovides a filtering process for the deductions and knowledge transferbetween levels. This helps “stabilise” and reduce context sensitivity.It also permits a strong method for checking the validity of componentconstruction. However, a price is paid: the filtering between layersimplies that the potential of each agent to contribute to the Gestalt isconstrained. A particular agent may be able to undertake a variety ofrelevant deductions but these may be “strained” or “filtered” as theagent passes its knowledge through an ontology layer. Hence thetheoretical full potential of the Gestalt is never actually realisable.

[0407] In order to overcome the above constraint in a sensible,practical and useful manner, it is necessary to review continuously theontology layers in the search for bringing new relationships and objectsinto “first class” status so that it may become part of the ontologyitself. That is, lessen the filtering process in a controlled manner. Todo so however, requires much thought since an incorrect change in anontology level can wreak havoc with the Gestalt operation. Of course itis possible to pass richer knowledge statements by using attributesthrough the ontology layers. However, it becomes the user'sresponsibility to ensure that the receiving agents can make sense of theadditional attributes.

[0408] 10.1 AAFID

[0409] Researchers at Purdue University have designed and implemented anagent-based architecture for Intrusion Detection, called AAFID(Autonomous Agents for Intrusion Detection) [Spafford, E and Zanboni,D., “Intrusion detection using Autonomous Agents”, Journal of computerNetworks, v34, pages 547-570,2000]. This architecture is based around afundamental paradigm of distributed computation. One or more softwareagents run on each protected host of a network, communicating any eventsof interest to a single “Transceiver” running on the host. Thiscomponent can perform some host-level fusion of alerts, but principallyexists to forward significant observations to a “Monitor” process, whichhas an even broader purview.

[0410] This architecture at first appears to have similarities to theapproach described herein, in that it supports multiple autonomousentities (each with a particular field of expertise) arranged in adistributed structure with hierarchy-based filtering. The AAFID system,however, does not appear to have a concept of multiple abstractionlayers—all agents, transceivers and monitors all reason within a singleuniverse of discourse which, apparently, contains both low-level andfairly high-level concepts.

[0411] Furthermore, the operation of these various entities seems tofocus purely on a data driven model; there is no obvious scope for usersto set goals for components, nor to directly query the internalknowledge state of the system. AAFID's hierarchical structuring ofagents seems limited to a single rooted tree, as opposed to our system'ssupport for generalised directed acyclic graph structures. There is alsono obvious scope for possiblistic or probabilistic reasoning within theAAFID architecture coupled with orthogonal semantic ontology layers.

[0412] 10.2 Comparison with the Bass' Comments

[0413] The following discussion providing some background to theinvention is intended to facilitate a better understanding of theinvention. However, it should be appreciated that the discussion is notan acknowledgment or admission that any of the material referred to waspublished, known or part of the common general knowledge as at thepriority date of the application.

[0414] In an edition of the Communications of the ACM “IntrusionDetection Systems & Multisensor Fusion: Creating Cyberspace SituationalAwareness”, in Communications of the ACM 43(4), April 2000 Bassspeculates on the future architecture requirements for IntrusionDetection Systems. In particular, he discusses the need for dataabduction and points to the requirement for three main levels ofsemantic ascension.

[0415] The Shapes Vector architecture shows some necessaryimplementation strategies and architectural modifications in order toachieve that goal state. In particular Shapes Vector views the conceptascension requirement as a continuum where at any point in the AI agentGestalt one “sees” knowledge production on looking “up”, and data supplylooking “down”. The three main levels in the Shapes Vector Gestalt aredelineated by the methods and logics used (ie. first order predicate,possiblistic and probabilistic), rather than some delineation as towhether there is information, data, or knowledge as depicted in FIG. 13.Bass requirements for a “discovery module etc” become less important inthe Shapes Vector architecture as any such function is a pervasive partof the system and is distributed amongst more primitive functions. TheAgent Gestalt feeds the visualisation engines rather than some specificevent though earlier papers do tend to indicate a separate module and assuch those papers are a little misleading.

[0416] 11. A Multi-Abstractional Framework for Shapes Vector Agents

[0417] The Shapes Vector Knowledge Architecture (SVKA) is intended toprovide a framework, in which large numbers of Intelligent Agents maywork collaboratively, populating layers of a highly ordered “Gestalt”.Previous definitions of the SVKA have focussed primarily onmacro-aspects of the architecture, describing a system in which eachlayer of the Gestalt represents a distinct universe of discourse asdescribed by the ontology associated with it.

[0418] Experience with building collaborative Intelligent Agent systemsfor Shapes Vector has highlighted the desirability of a more flexiblemodel, one that allows for the subdivision of these “ontology layers”into a number of sub-layers. Each sub-layer in such a divided modelshares a common universe of discourse (i.e., all reference a commonontology). Intelligent Agents can populate any of these varioussub-layers, allowing for the construction of systems capable of verygeneral forms of data fusion and co-ordination.

[0419] Furthermore, it is envisaged that future requirements on the SVKAwill involve the necessity of maintaining several “parallel universes ofdiscourse” (e.g., running a sub-Gestalt in the domain of Security inparallel with another sub-Gestalt in the domain of EM Security). Suchparallel universes may have entry and exit points into one another (atwhich appropriate translations take place). They may furthermore sharesimilar abstractional levels, or may even overlap the abstractionallevels of multiple other universes.

[0420] In order to satisfy these two demands, the SVKA definitionrequires elaboration. In this paper we undertake a redefinition of thearchitecture which expands it in a number of ways to meet theserequirements. Key features are:

[0421] The SVKA Gestalt is divided into an arbitrary number of Locales,

[0422] A Universe of Discourse and an Instance Number identify eachLocale,

[0423] Each Locale contains a number of levels at which IntelligentAgents may reside.

[0424] A Locale may optionally nominate a single entry point: a remotelocale and a level within that locale, from which input data is receivedinto the locale,

[0425] A Locale may optionally nominate a single exit point: a remotelocale and a level within that locale, to which output data is sent fromthe locale,

[0426] 11.1 Concepts

[0427] The Shapes Vector Knowledge Architecture (SVKA) contains exactlyone Shapes Vector Gestalt Framework (SVGF) or “Gestalt”. The Gestalt isan abstract entity in which groups of collaborating software agents maybe placed.

[0428] The Shapes Vector Gestalt Framework contains an arbitrary numberof Shapes Vector Gestalt Locales (SVGLs) or “Locales”. A Locale is anabstract entity. in which hierarchies of collaborating software agentsmay be placed. The defining characteristic of a Locale is that it isintimately tied to exactly one Universe of Discourse (UoD). For each UoDthere may be multiple Locales simultaneously active, thus to distinguishthese we also tag each Locale with an instance ID. This is unique onlywithin the context of all Locales tied to the same UoD. For examplethere can exist a Locale with UoD “Level I Cyber Ontology”, instance 0simultaneous with a Locale with UoD “Level 2 Cyber Ontology”, instance0. However two Locales with UoD “Level 1 Cyber Ontology” and instance 0cannot co-exist.

[0429] Each Shapes Vector Gestalt Locale is divided into an arbitrarynumber of Shapes Vector Gestalt Locale Levels (SVGLLs) or “Levels”. ALevel is an abstract entity in which a non-cooperating set of agents maybe placed. Each Level has a unique Level Number within the Locale (azero or positive real number); Levels are notionally ordered into asequence by their Level Numbers.

[0430] In addition to a UoD and instance ID, each Locale also optionallypossesses two additional attributes: an entry point and an exit point.Each refers to a Level of a remote Locale, that is each referencecontains the UoD and instance number of a Locale not equal to thisLocale, and also nominates a particular (existent) Layer within thatLocale. The entry point of a Locale defines a source of data, which maybe consumed by the agents at the lowest Level of this Locale. The exitpoint of a Locale defines a destination to which data generated byagents in the highest Level of this Locale may be sent.

[0431] It is specifically forbidden for Locales within the Gestalt to beat any time directly or indirectly arranged in a cycle via their entryand/or exit points. That is, it must be impossible to draw a path fromany point in any Locale back to that same point utilising entries andexits between Locales.

[0432] A Shapes Vector Gestalt Locale, which is divided into n Levels,contains n−1 Shapes Vector Assertion Routers (SVARs) or “AssertionRouters”. An Assertion Router is a concrete software entity whichreceives input from one set of agents, performs some semantic-basedfiltering, then forwards the relevant sub-sections on to each of a setof agents in a second (disjoint) set. Each Assertion Router has anassociated Level Number unique within the Locale (a zero or positivereal number); Assertion outers are notionally ordered into a sequence bytheir Level Numbers. Furthermore, each Assertion Router has an InstanceID (a zero or positive integer) Furthermore, which is globally unique.

[0433] There is a one-to-one mapping between Locale Level Numbers andAssertion Router Level Numbers, defined by the following relationship.The Assertion Router with Level Number n receives input from agentspositioned in Locale Level Number n and provides output to agents whichare resident at the next Locale Level Number after n.

[0434] A Shapes Vector Intelligent Agent (SVIA) or “agent” is a concretesoftware entity which resides in exactly one Level of exactly one Localewithin the Gestalt. Agents that reside above the lowest Level of theLocale may (optionally) receive input either from a direct source, orfrom the Assertion Router within the Locale which has the next lowestLevel Number (or both). An agent that resides at the lowest Level ofLocale may (optionally) receive input either from a direct source, orfrom the Assertion Router present in the entry point remote Locale (ifone was specified) which has a Level Number equal to the Level definedin the entry point specification (or both).

[0435] Agents which reside below the highest Level of the Locale may(optionally) provide output to either a direct sink, or to the AssertionRouter with Level may (optionally) provide output to either a directsink, or to the Assertion equal to its own (or both). An Agent thatresides at the highest Level of Router present in the exit point remoteLocale (if one was specified) which has a Level Number equal to theLevel defined in the exit point specification (or both).

[0436] An agent may never receive input from the same Assertion Routerto which it provides output.

[0437]FIG. 12 illustrates these concepts for a single Locale Gestalt of4 Levels, while FIG. 13 shows a more comprehensive example.

[0438] 12. Summary

[0439] The knowledge-processing elements of the Shapes Vector systemincorporate a broad variety of tools and techniques, some novel and sometraditional, which combine to enact a flexible and powerful paradigm formulti-abstractional reasoning. The central feature of the approachdisclosed herein is the methodology of bridging broad semantic gaps (inthe embodiment described that is illustrated, from very simpleobservations about a computer network to high-level statements about thestate of that network) by decomposition into a series of abstractionlayers. This specification describes this layered architecture and alsoprovides details about the forms of abstraction provided at the firstthree layers. These include epistemic logics for possiblistic reasoning(at level 2) and probabilistic reasoning (at level 3).

[0440] The key feature of the disclosed knowledge architecture thatavoids difficulties of context sensitivity and ambiguity is its simpleset of structuring rules. These provide strict guidelines for placementof agents within abstractional layers and limit the patterns ofcommunication between agents (preferably prohibiting intra-levelcommunication as well as insisting on passing through an ontologybetween layers).

[0441] Experience with building and using the Intelligent AgentArchitecture it has shown it to be highly flexible, with theincorporation of “foreign” knowledge processing tools into the ShapesVector Gestalt proving a “simple” exercise. The architecture has alsoshown itself to provide great potential for approaching knowledge-baseddeductive solutions to complex problems not only in the domain ofcomputer security but also in many other domains, both related andunrelated.

[0442] The Intelligent Agent Architecture features specifically include:

[0443] 1. An abstraction hierarchy with multiple layers separated byformal ontologies.

[0444] 2. Three particular abstraction layers of interest are thoseconcerned with first-order logic, possibilistic logic and probabilisticlogic.

[0445] 3. Agents located within a layer of the architecture areprohibited from interacting with agents within the same layer (ie. Nopeer-to-peer communication).

[0446] 4. Agents located within a layer of the architecture maycommunicate with agents located in the layer immediately below thatlayer (if such exists) and/or agents located in the layer immediatelyabove the layer (if such exists).

[0447] 5. The architecture may incorporate a Knowledge Base in whichpersistent information resides.

[0448] 6. Communication between agents must always be represented interms of the ontology sandwiched between the sender and receiver'slayer. Communications must be context-free with respect to thatontology.

[0449] 7. Agents within the architecture may operate across atime-window, ie, a temporal region of current consideration. A user maydynamically alter parameters of an agent's time-window.

[0450] 8. Third party knowledge processing tools (agents) may be easilywrapped and inserted into the architecture. The ontologies presentwithin the framework ensure that only relevant knowledge transfer takesplace between such elements and other agents.

PART 3 DATA VIEW SPECIFICATION

[0451] 1. Data View Specification

[0452] Data View is briefly discussed in Part 1 Section 3.3 of theShapes Vector Overview, in this specification. The following is apreferred specification of its characteristics.

[0453] 1.1 Universe

[0454] a universe has a variable maximum radius and contains any numberof virtual objects.

[0455] there may be multiple universes.

[0456] the number of universes can be dynamically adjusted using append,insert, and delete operations specified via the user (human orappropriately programmed computer).

[0457] universes are identified by unique names, which can be eitherauto generated—a simple alphabetic sequence of single characters, or canbe specified by the user when appending or inserting universesdynamically.

[0458] to assist in simplifying the display, nominated universes can betemporarily hidden. However all force calculations and position updatescontinue to occur. Hidden universes are simply temporarily ignored bythe rendering phase of the application. A universe is then not humanobservable.

[0459] a universe can be represented as a two-dimensional plane (in theembodiment a circle), but it is subject to selective zoom and syntheticstrobes in a similar fashion to Geo View which may provide a thirddimension elevation.

[0460] there are at least two possible starting states for a universe:

[0461] the big bang state in which all objects are created in the centreof the universe;

[0462] the maximum entropy state in which all objects are evenlydistributed around the maximum radius of the universe.

[0463] a universe maybe rendered with a circular grid, with identifyinglabels placed around the perimeter. The number of labels displayedequidistantly around the perimeter can be specified statically ordynamically meaning those labels can be fixed or move in concert withother changes.

[0464] multiple universes are rendered vertically displaced from eachother. Inter-grid separation can be dynamically changeable via a controlmechanism, such as a socket to be discussed later.

[0465] separation between grids can be specified either globally usinginter-grid size or for specific grids as a distance from adjacent grids.

[0466] different universes can have different radii, and their grids canbe drawn with different grid sizes and colours.

[0467] all initial settings for grid rendering are to be specifiedthrough the MasterTable. These include grid size, inter-grid separation,grid colour, and grid (and hence universe) radius for each universe.

[0468] grid settings (radius, number of radii, number of rings,intergrid spacing) can be altered dynamically via the user.

[0469] object positions are clamped to constrain them within theuniverse radius.

[0470] As a result of this:

[0471] When an object located at the edge of the universe experiences arepulsive force that would place it outside the universe (forces betweenvirtual objects will be discussed later in the specification), theobject is constrained to stay within the universe so that the objectslides along the rim of the universe away from the source of therepulsive force. Forces that tend to draw objects away from the rimtowards the interior of the universe result in typically straight-linemotion towards the source of attraction.

[0472] the user may specify which virtual objects or sets of virtualobjects are in a particular universe using an object selector, and thismay dynamically change using append or replace operations on existingspecifications.

[0473] if a user replaces the specification of the destination universefor objects matching a particular object selector, then the objects willmove from the universe they were originally placed in as a result ofthis specification to the new universe. Likewise if a user appends a newdestination universe specification, then all objects in existence thatmatch the associated object selector will appear in the new universe inaddition to wherever they currently appear.

[0474] in all cases where objects are moved between or duplicated touniverses, all force interactions, phantoms, interaction markers andradius of influence displays will be updated to reflect this fact.

[0475] Force interactions are updated so they only occur between objectsin the same universe.

[0476] Phantoms are moved/duplicated along with the parent primaryobject.

[0477] Interaction markers are moved/duplicated to remain connected tothe object.

[0478] Radius of influence displays are duplicated if necessary.

[0479] 1.2 Objects

[0480] an object has a set of attributes (consisting of name, valuepairs) associated with it.

[0481] an object has a two sets of references to other objects withwhich it interacts, named its mass interaction set and chargeinteraction set. Events or other external mechanisms modify these twosets.

[0482] an object can have further sets of references to other objects.These sets have names specified at run-time by events and can be used tovisualise further interactions with other objects using markers (seesection 1.7 in this part of the specification).

[0483] an object can have further sets of references to other objectsthat are used in building aggregate objects—see section 1.3 of this partof the specification for details.

[0484] an object stores values for mass and charge for each flavour (aterm explained later in the specification) it possesses.

[0485] an object may inhabit one or more universes, and thisrelationship can be displayed using markers.

[0486] 1.3 Aggregate Objects

[0487] objects can be aggregated to form a composite.

[0488] each aggregate object has one primary (the parent or container)object, and zero or more secondary objects (the children or containees).

[0489] aggregate objects cannot aggregate hierarchically.

[0490] determination of container-containee relationships occurs on thebasis of “contains” and “contained-in” network object attributes. Theserelationships are stored in a database and are always kept up to dateand consistent with the latest known information. This means any newinformation overrides pre-existing information. For example:

[0491] If an attribute indicates A contains B, then it must be ensuredthat all relationships where B is a container are removed from thedatabase as they are no longer valid since B is now a containee. Thesame attribute A contains B also indicates that A can no longer be achild of another object, since it is now a container, and so all thoserelationships are removed from the database. Finally the relationships“A contains B” and “B is containee of A” are added to the database.

[0492] to avoid processing overheads, the actual relationships ofobjects in the display are not updated to reflect the state of therelationship database until an object is re-instantiated—usually bybeing moved/duplicated to another universe.

[0493] the aggregate object is treated as a single object for thepurposes of force and velocity determination, interaction marker, radiusof influence, and phantom displays (subject to the considerations setout below).

[0494] when a new object comes into existence in a universe (either asresult of an event being received, or as result of dynamic adjusting ofdestination universe specifications), it can either become a primary ina new aggregate group, or enter the universe as a secondary in apre-existing group depending on the containment/containee relationshipsin force at the time.

[0495] if a new object enters a universe as a primary in a newly createdaggregate, it will attempt to determine which other objects in theuniverse should be adopted to become secondaries. The adoption occurswhen another object (potential adoptee) is located that is a containeeof the new object (according to the relationship database).

[0496] When the potential adoptee is a primary in an aggregate withsecondaries, the secondaries are evicted before adopting the primary.The evicted secondaries are now inserted into the universe using theinsertion policy in force, and they in turn determine potential adopteesand adopt where possible as described.

[0497] the summed masses and charges (section 1.5 in this part of thespecification) of all objects within an aggregate are used forforce/mass calculations.

[0498] each individual element in an aggregate maintains it's own mass,and masses of like flavour are summed when determining the mass of anaggregate.

[0499] an aggregate object maintains and decays a single total chargefor each flavour.

[0500] When an object joins an aggregate its charges are added to thesummed charges of like flavour for the aggregate. When an object leavesan aggregate no change is made to the summed charge as it cannot beknown (because of charge decay) what proportion of the total charge isdue to the object in question.

[0501] when an object receives additional charge as result of an event,the new charge is added to the total for the aggregate containing it.

[0502] if any object within an aggregate object displays a mass orcharge radius of influence (see section 1.9 in this part of thespecification), the mass/charge radius is displayed for the entiregroup, provided the group as an entity has a non-zero mass or charge ofthe flavour as specified by the radius of influence definition.

[0503] display of phantoms (section 1.7 in this part of thespecification) of aggregate objects is driven only by the primaryobject. The phantoms appear as duplicates of the primary object andtrail the primary object's position. If an object matches a phantomingspecification, but it happens to be a secondary within an aggregateobject then no action is taken.

[0504] if an object (A) within an aggregate is required to displayinteraction markers (see section 1.8 in this part of the specification),the interaction markers are drawn from the primary of the aggregateobject containing A to the primary of the aggregate(s) containing thedestination object(s).

[0505] In addition, when interaction markers are drawn in response topicking of an object, they are drawn from the primary of the aggregatecontaining the picked object to all duplicates in other universes of alldestination objects that are in the relationship to the source objectthat is being visualised by the markers.

[0506] when interaction markers are drawn in response to the matching ofan ObjectSelector, markers are drawn from all duplicates of the sourceaggregate object to all duplicates of the destination aggregateobject(s).

[0507] when interaction markers are used to highlight duplicates of thesame object in multiple universes, a single multi-vertex marker isdisplayed which starts at the duplicate appearing lowest in the stack ofuniverses and connecting all duplicates in order going up the stack andending at the highest appearing duplicate in the stack.

[0508] 1.4 Object Selector

[0509] An object selector specifies a set of objects using setexpressions involving set union (+), set difference (−), and setintersection ({circumflex over ( )}) operators. The intersectionoperator has the highest precedence, difference and union have equallower precedence. Parentheses can be used to change operator precedence.The set operations have the following operands:

[0510] all—set of all objects;

[0511] class(classname)—set of objects in a given class;

[0512] objects(predicate)—set of objects satisfying a predicateexpressed in terms of boolean tests (using and, or, not) on attributevalues (e.g. objects(ram>=128000000 && type=sun)) and existence ofattributes (e.g. objects( attributes( attributename, attributename, . .. )));. The “and” &&) and “or” (∥) operators have equal high precedence.The “not” operator (!) has lower precedence. Parentheses can be used tochange operator precedence

[0513] Flavor(flavorname)—set of objects having all attributes in thegiven flavour's definition.

[0514] instance(objectid)—set containing object with given object id.

[0515] Object selectors are named when defined. This name is used as ashorthand means of referring to the object selector without having torepeat its definition, for example when defining an action on the basisof an object selector.

[0516] Object Selectors can be defined via a control port, or via astart-up setting, currently stored in the ApplicationSettings part ofthe MasterTable file.

[0517] 1.5 Mass, Charge and Flavours

[0518] There exist different flavours of mass and charge.

[0519] A flavour is defined by the user as a collection of five-tuples,each defining the flavour with respect to a particular class of objects.The tuple consists of flavour name, object class (or all), attributesexpression-listing attributes which must exist, formula for mass andformula for charge. There may be multiple such tuples with the sameflavour name, which together define a flavour for multiple classes ofobjects. The formulae are used to calculate the amount of mass or chargeof the flavour, which the object possesses, and they are arithmeticexpressions involving object attribute value terms. Note that it is asemantic error for the attributes expression not to include attributeswhich feature in the mass or charge formulae. For example: Flavour {Strawberry, Computer, Attributes [runs ram data_rate_in data_rate_out],ram/1024, data_rate_in + data_rate_out; }

[0520] If the result of evaluating a mass formula is less than a smallpositive number (ε) then the value used for the mass in calculationsshould be ε.

[0521] An object may have an amount of flavoured mass or charge if thereis a corresponding definition of the flavour for the class of object andthe object satisfies the attributes expression for that flavour.

[0522] Charge may be set to decay (at a particular rate) or accumulate(ie. a rate of zero) on a per flavour basis. The decayfunction can beset to one of a fixed set (exponential, linear and cosine) on a perflavour basis.

[0523] Mass and charge may each have a radius of influence specified ona per flavour basis. Objects that fall within the respective radii of anobject may generate a force on the object as a result of their mass orcharge respectively, and objects that lie outside this region have noinfluence on the object.

[0524] The radii of influence for objects may be graphically depicted atthe user's discretion. Note that multiple radii may apply due todifferent radii for mass and charge and for different flavours of thesame.

[0525] When an event arrives that affects one of the attributes listedin a flavour definition, then the object's mass and charge are to berecalculated using the arithmetic expressions specified in said flavourdefinition. The newly calculated mass will replace the existing mass,and the newly calculated charge will be added to the existing charge.

[0526] In addition there are special considerations relating tomass/charge/flavour and aggregate objects. See section 1.3 in this partof the specification regarding those.

[0527] 1.6 Forces

[0528] There are two types of forces acting on objects in the universe,gravitational (as a result of the mass of an object) and electrostatic(as a result of the charge on an object).

[0529] The gravitational force is repulsive and the electrostatic forceis attractive.

[0530] Forces are two-dimensional vectors and are additive.

[0531] The velocity of an object is proportional to the force acting onit divided by its mass (ie. acceleration is disregarded). Note thatflavours need to be taken into account in this calculation.

[0532] There is a variable maximum velocity which applies to all objectsin a universe.

[0533] Only masses and charges of the same flavour may produce aresultant force.

[0534] The velocity due to gravitational forces of an object iscontributed by the gravitational forces which result from each of theobjects in its mass interaction set (using the mass value for eachrelevant flavour) which are also within the radius of influence formass. These forces are divided by the correspondingly flavoured mass ofthe object to arrive at velocities.

[0535] The velocity due to electrostatic forces of an object iscontributed by the electrostatic forces which result from each of theobjects in its charge interaction set (using the charge value for eachrelevant flavour) which are also within the radius of influence forcharge. These forces are divided by the correspondingly flavoured massof the object to arrive at velocities.

[0536] The net velocity of an object is the sum of the gravitational andelectrostatic velocities for that object.

[0537] 1.7 Phantoming

[0538] When selected, objects will be able to display a history ofprevious positions by displaying a ‘phantom’ at a number of previouslocations.

[0539] Objects for which phantoms are drawn are selected using namedObject Selectors.

[0540] Display Parameters for phantoming can be set in theApplicationSettings part of the MasterTable or via a control socket, andare associated with a previously defined Object Selector.

[0541] Display parameters include the time spacing between phantoms, thedisplay style (eg transparent or wire frame), and the number of phantomsto show.

[0542] Multiple Object Selectors and associated display parameters canbe used to display any desired combination of phantoms.

[0543] In addition there are special considerations relating tophantoming and aggregate objects—see section 1.3 in this part of thespecification for those.

[0544] 1.8 Markers

[0545] Interaction markers may be used to highlight interaction betweenweakly interacting objects.

[0546] Interaction markers make use of named Object Selectors todetermine which objects have interaction markers displayed.

[0547] Interaction markers may span multiple universes.

[0548] Display parameters for markers can be set in theApplicationSettings part of the MasterTable or via the control socket,and are associated with an interaction type.

[0549] Display parameters for markers include line style, width andcolour. Each interaction type is drawn with its own independentlyspecified line style width, and colour.

[0550] Marker—the user intellectively picking an object on the displaycan optionally toggle the display.

[0551] Multiple Object Selectors and associated display parameters canbe used to display any desired combination of markers.

[0552] In addition there are special considerations relating tointeraction markers and aggregate objects—see section 1.3 in this partof the specification for those.

[0553] 1.9 Radius of Influence Display

[0554] Radius of influence can be visualised for selected objects astransparent disks.

[0555] Objects for which radius of influence are displayed are selectedusing the Object Selector mechanism.

[0556] Display parameters include which flavour to display the radiusfor, whether the charge or mass radius is to be displayed, the colour ofthe displayed disk, and the transparency level of the displayed disk.

[0557] Display parameters can be set in the ApplicationSettings part ofthe MasterTable or via the control socket, and are associated with apreviously defined Object Selector.

[0558] Multiple Object Selectors and associated radius of influencedisplay parameters can be in use simultaneously to display any desiredcombination of radii.

[0559] In addition there are special considerations relating to radiusof influence displays and aggregate objects—see section 1.3 in this partof the specification for those.

[0560] 1.10 Pulses

[0561] A flavoured pulse of charge or mass may be applied at anylocation within a universe, and has influence over the entire universe.That is, a flavoured pulse is applied without regard to the mass andcharge radii associated with its flavour.

[0562] 1.11 Irregular Functions

[0563] A user may “shake” a universe at any particular moment, byeither:

[0564] perturbing each object by a random amount (under a variablemaximum)

[0565] ;—randomly placing each object within the universe.

[0566] A user may reproduce the start state of a universe at anyparticular moment, as at least either the big bang or maximum entropystate.

PART 4 GEO VIEW SPECIFICATION

[0567] 1. Introduction

[0568] 1.1 Identification

[0569] This document relates to the GeoView Module for the VisualsSub-system of Shapes Vector.

[0570] 1.2 System Overview

[0571] 1.2.1 General

[0572] Shapes Vector is a system, which in the embodiment used toillustrate its principles and features provides an analyst or systemadministrator with a dynamic real-time visualisation of a computer ortelecommunications network. Shapes Vector is an advanced 3-D graphicalmodelling tool developed to present information about extended computersystems. Shapes Vector places users within a virtual world, enablingthem to see, hear and feel the network objects within their part of thisworld. The objects may be computers, files, data movements or any otherphysical or conceptual object such as groups, connected to the systembeing navigated. Seeing an object with a particular representationdenotes a class of object and its state of operation within its network.

[0573] Just as familiar objects make up our natural view of the physicalworld, so too a computer network is made up of physical objects, such ascomputers, printers and routers, and logical objects, such as files,directories and processes.

[0574] Shapes Vector models network objects as 3D shapes located withinan infinite 3D universe. An object's visual attributes, such as colour,texture and shading, as well as its placement and interaction with otherobjects, provide the user with significant information about the way thereal network is functioning.

[0575] 1.2.2 Geo View Module Scope

[0576] GeoView, along with DataView, is one of the ways in which theuser can view, and interact with, the data produced by the AgentsSub-system. Each of these views is defined by certain characteristicsthat allow it to provide a unique representation of the data. GeoViewhas an emphasis on the physical objects with a geographic perspective.This means it places a heavy importance on objects related to thephysical rather than the logical world, and that these objects tend tobe laid out in a traditional geographic manner.

[0577] While objects such as computers, printers and data links are ofprime importance to GeoView, logical objects such as network traffic,computer processors, and user activity are also displayed because of therelationships between the physical and logical objects.

[0578]FIG. 1 shows the library dependency relationship between theGeoView module and other modules. The full relationship between theSub-systems and between this module and other modules of thissub-system, is shown in the System/Sub-system Design Description.

[0579] 1.3 Overview

[0580] This part of the specification provides a Detailed Design for theGeoView Module of the Visuals Sub-system (CSCI) of the Shapes VectorProject.

[0581] This module encompasses the following sub-components:

[0582] Layout Hierarchy

[0583] Layout Structure Template Library (LSTL)

[0584] The content of this part is based on the Data Item Description(DID) DI-IPSC81435, Software Design Description (SDD), from the USMilitary Standard MILSTD-498 [1] using MIL-STD-498 Tailoring Guidebook[2] for the development project.

[0585] Detailed design information in this document is based on thetechnical content of other parts of this specification.

[0586] 2. Referenced Documents

[0587] 2.1 Standard

[0588] [1] MIL-STD498, Military Standard—Software Development andDocumentation, US Department of Defence, Dec. 5, 1994.

[0589] [2] MIL-STD498 Overview and Tailoring Guidebook, Jan. 31, 1996.

[0590] 3. Module-wide Design Decisions

[0591] This section presents module-wide design decisions regarding howthe module will behave from a user's perspective in meeting itsrequirements, and other decisions affecting the selection and design ofthe software components that make up the module.

[0592] 3.1 Design decisions and goals of Geo View

[0593] GeoView is designed with only one executable.

[0594] GeoView has logically divided sub-components: Layout Hierarchyand Layout Structure Template Library (LSTL).

[0595] Unless otherwise specified, the programming language used inGeoView is C++. The following list identifies the design concepts thatcharacterise GeoView:

[0596] 1. Physical: Due to the focus on the physical world, moreimportance is placed on physical objects and less importance on logicalobjects.

[0597] 2. Geographic: The default mechanism for placing objects in theworld is to map them according to a physical location. If this is notpossible then other methods must be used.

[0598] 3. Shape: Shape is used to identify unique objects. Differentobject types will have different shapes while similar object types willhave similar shapes.

[0599] 4. Motion: Movement of objects (translation, rotation) typicallyrepresents activity in the world. A moving packet represents trafficflow while a spinning process shows that it is active.

[0600] 5. Sound: Sound is linked to movement of objects or a change invisual appearance or a change in state.

[0601] 6. Feel: Feel or touch senses can be used to provide additionalemphasis to events linked to movement of objects where for exampleobject come into close proximity suddenly.

[0602] 4. Module Architectural Design

[0603] At the architectural level, the sub-systems (CSCJs) aredecomposed into modules. At the detailed design level, the modules aredecomposed (if applicable) into executables or libraries. This sectiondescribes the module architectural design.

[0604] The major architectural decomposition of GeoView comprises thefollowing component:

[0605] GeoView General (Section 4.1.1 of Part 4)—contains all theclasses for the support of the View, including interfaces withWorldMonitor for handling incoming events from Tardis, MasterTable forinput and output of master tables, and LayoutHierarchy for handling thelayout of network objects within the world.

[0606] and sub-components:

[0607] LayoutHierarchy (Section 4.1.2 of Part 4)—responsible for thenode store and data structure of GeoView. It places graphical(renderable) objects into the scene graph based on layout rulesspecified in the MasterTable. It uses the LSTL to manage structurenodes; and

[0608] LSTL (Section 4.1.3 of Part 4)—responsible for placing networkobjects (nodes) into layout structures such as rings, stars, and lines.The LSTL is a generic library with its components being templates. Thelayout structures covered include:

[0609] Tree, Graph, Line, Star, Matrix, Rectangle and Ring.

[0610]FIG. 15 shows an overview diagram for GeoView. Thecomponent/sub-components are described at an architectural level in thesections below. Detailed design of the sub-cmponents LayoutHierarchy andLSTL is included in Section 5. GeoView uses the Layout StructureTemplate Library (LSTL) framework by instantiating it with theLayoutHierarchy node type.

[0611] 4.1 Geo View Functional Design

[0612] 4.1.1 Geo View General

[0613] This section is divided into the following architectural designsub-sections:

[0614] Event Handling

[0615] MasterTable Functionality

[0616] CCI Interface

[0617] GeoView Processing and Caching

[0618] 4.1.1.1 Event Handling

[0619] The World Monitor receives events describing NetworkObjects fromthe Tardis process, via shared memory as shown in FIG. 15. Eachrecognised network object has its own event type, with event typescoming in the six variants shown in Table 1. TABLE 1 Network ObjectEvent Handlers Event Type Variant Event Handler Functionality AddObjectCreate a new NetworkObject AddObjectAttributes Add new attributes to anexisting NetworkObject ReplaceObject Replace a NetworkObject completelyReplaceObjectAttributes Replace a NetworkObject's attributesRemoveObject Temporarily removes a NetworkObject (tagged for deletionwhich cannot be undone) Purge zombies Permanently removes aNetworkObject

[0620] Currently the handlers for AddObject and AddObjectAttributes areidentical, both adding an object if none exists, and merging in theassociated attributes as well.

[0621] 4.1.1.2 MasterTable Functionality

[0622] The MasterTable is a hierarchical repository used for mappingnetwork object attributes to visual attributes. In operation, when avisual attribute is required, an address is constructed using theapplication name, the object type, and the network object. TheMasterTable is queried using this address and a list of matchingattributes is returned. These could include direct attribute settings,or attribute tests, where for example an object might become aparticular colour if the value of a specified network attribute of theNetworkObject in question is greater than a given constant. TheMasterTable also contains the Layout Rules determining whatlayout-structure objects are placed in.

[0623] 4.1.1.3 CCI Interface

[0624] The Component Control Interface (CCI) consists of textualcommands sent via a socket that can be used to drive various portions ofthe application. Each Shapes Vector process usually has one or moreCCIs, with each CCI being attached to a logically distinct portion ofthe application. For example, GeoView has a CCI for controlling therenderer, one for the SVWorld (shown as Virtual World on FIG. 1), whichcurrently deals mainly with selective zoom and object selectors, and onefor the World Monitor that gives access to commands relating toprocessing of incoming events.

[0625] When a command arrives on the CCI socket, the CCI thread notifiesthe main thread that it wants access to the mutex. The next time themain thread checks (in SV_View::postTraverseAndRender) it will yield tothe CCI thread. It then performs all necessary processing beforerelinquishing the lock.

[0626] The notification mechanism is embodied in a class calledControlMutex that resides in the svaux library. It allows a higherpriority thread to simply check the value of a flag to see if anyone iswaiting on a mutex before relinquishing it, rather than performing acostly check on the mutex itself. Currently the ControlMutex is not usedin processing, rather the CCI is checked once per renderer cycle inSV_View (the base view).

[0627]FIG. 16 shows the processing within the CCI thread as part of theGeoView thread diagram.

[0628] 4.1.1.4 GeoView Processing and Caching

[0629] The basic structure of processing in GeoView is for externalevents to arrive detailing the existence of world objects, facts knownabout them and their relationships to other world objects in the form ofattributes. The virtual depiction of these real world objects occursgraphically via leaf nodes in the GeoView universe where visualattributes for each type of possible world object is specified via theMaster Table. Such visual attributes may be specified on the basis ofattributes of the world objects.

[0630] In order to determine where to place these leaf nodes, the layoutrules from the MasterTable are used. These layout rules specifyattachment, containment and logical groupings of leaf nodes on the basisof their attributes.

[0631] To enable this, the building of the GeoView world is broken intothe following five phases, with phases 2-5 referred to as the Layout′process:

[0632] 1. Event Insertion and Removal

[0633] Add or remove attributes arrive encapsulated within networkobjects via events.

[0634] New leaf nodes in the layout hierarchy are created wherenecessary attributes and inversed attributes are added to, or removedfrom, leaf nodes.

[0635] Caching for the next four processing steps (i.e. Layout) occurs.

[0636] 2. Leaf Building

[0637] Graphical objects are built according to their visual mapping'sin the MasterTable.

[0638]

[0639] 3. Layout Rule Application

[0640] Layout-structure, attached-to and located-in rules from theMasterTable are applied to the objects. Any necessary grouping layoutstructures are created and parent-child relationships are formed thatwill dictate positioning, attachment, and containment.

[0641] 4. Leaf Edge Creation

[0642] New edges between leaf nodes are created.

[0643] 5. Object Relative Placement

[0644] Both leaf nodes and structure nodes are placed on the basis oftheir hierarchical relationships (attachment and containment) in thelayout hierarchy as well as their parent structure nodes.

[0645] For efficiency, each phase of processing is associated withcaching certain operations such that repeat operations do not occur. TheGV_WorldMonitor turns off the layout flag before the insertion of abatch of events into the root of the GeoView layout hierarchy After abatch of events has been inserted, the layout flag is turned back on,and the entire batch is processed fully This procedure represents thestart point for caching optimisation in layout hierarchy

[0646] If such a caching strategy were not employed, processing time maybe greatly affected. This is due to relative placement using boundsinformation. For example, consider Computer A with 20 child modems, eachof them attached-to it. As the substructure for attached-to objects (forexample ring) is called for EACH of the child modems, a bounds radiuschange will occur Since this may affect the overall compositestructure's bounds radius (i.e. the parent Computer A's bounds) then therelative placement algorithm must be called for the Computer A's parentstructure. This occurs in a recursive manner to the root layoutstructure. Without caching, the relative placement of Computer A wouldneed to occur a minimum of 20 times. By caching the fact that Computer Arequires placement (on level base—2 of the layout hierarchy tree) it canbe guaranteed that Computer A's relative placement is called a maximumof once only processing this occurs. TABLE 2 provides an overview ofwhich operations are cached and at what stage of Level ProcessingCaching 1 Object insertion and removal Each leaf node created or altered2 Object building and layout Each structure node requiring placement 3Edge creation Leaf nodes requiring edge updates 4 Base level structureplacement Base Level-I structure placement, leaf nodes requiring edgeupdates 5 Base Level-I structure Base Level -2 structure placementplacement, leaf nodes requiring edge updates . . . . . . . . . N TopLevel Structure Placement Leaf nodes requiring edge updates N + 1 Leafnodes requiring edge updates

[0647] 4.1.2 LayoutHierarchy

[0648] The LayoutHierarchy is the view specific node store and system ofdata structures that implement GeoView. It consists of a hierarchy ofclasses that combine to form a logical object hierarchy of graphical andgrouping objects that parallel the structure of the Scene Graph.

[0649] 4.1.2.1 Class Hierarchy

[0650] The class relationship diagram for Layout Class Hierarchy isshown in FIG. 17

[0651] The Layout Hierarchy data structure is a hierarchical,arbitrarily Nodes represent either logical grouping constructs(i.e.depthed tree. LH_SNodes, LH_CoordGroupSet or LH_CoordGroups) orgraphical objects in the GeoView universe (i.e. LHLeaf).

[0652] All classes inherit from the abstract base class LU_Node. TheLU_Root, LH..EdgeSet and LWCoordGroupSet classes each representsingleton instances.

[0653] LH_Node inherits from Cfltem and contains the base interface forall classes in GeoView. This interface includes the ability to set andcheck placement flags that indicate the type of an object add and removeobjects from the scenegraph, look up operations in the MasterTable andplacement specific functions.

[0654] LH_Root provides the instance for the root of the Geo Viewuniverse and is the core interface to the Layout Hierarchy datastructure. It allows the insertion of events into the World, the look upof all nodes, the application of layout rules, object composition andplacement caching for speed up.

[0655] LH_Leaf represents the actual visual objects in GeoView. Itprovides an interface for altering any visual aspects of objects, aswell as an interface for maintaining any attachment or containmentrelationships an object may have.

[0656] LHEdgeSet maintains the set of connection lines in the World. Italso has provision for temporarily turning lines on and off.

[0657] LH_CoordGroupSet is a singleton container class forLH_CoordGroup(s) that is itself a container for layout structures and/orleaf objects. It groups those objects together for which specificlocation data is known, and maintains them positioned as close aspossible to that location.

[0658] LH_SNode is the abstract parent class for the layout structureclasses. It provides the interface to the layout structures allowinginsertion and deletion of objects, calling on the relative placement ofobjects and the maintenance of graph edges and location data for layoutstructures.

[0659] The layout structure classes themselves are grouping constructsfor leaf objects and provide the interface to the LSTL generic layoutstructures. These can be instantiated with generic objects specific toviews and the objects are placed on the basis of the traits particularto the layout structure (e.g. Star or Graph). Grouping rules arespecified in the MasterTable.

[0660] 4.1.2.2 Logical Object Hierarchy

[0661]FIG. 18 shows Logical Object View of the types of parent-childrelationships allowable in the data structure and how these classes worktogether to form the logical object view of the GeoView visual system,which is also hierarchical.

[0662] The three singletons of Root, CoordGroupSet and Top LevelStructure form level 0 and level 1 of the layout hierarchy tree.

[0663] A Root instance forms the parent of the tree that represents thelogical structure and interconnections between objects (LWLeaf) andgrouping constructs (children of LU_CoordGroupSet or LH_SNode in FIG.17). The interconnections of these objects and groups are mirrored inthe Scene Graph, which is the interface to the renderer visual system.

[0664] At the second level of the hierarchy there are two singletoninstances, viz, the Top Level Structure and the CoordGroupSet. Allobjects with location data or layout structures that have inheritedlocation data are placed into a CoordGroup within the CoordGroupSetrepresenting the geographic region given by the location. Visual objectswhen they first enter the world are placed into the Top Level Structure,which is the base layout structure of the world. CoordOroups containnodes with specific location data, whilst all nodes beneath the TopLevel Structure undergo relative placement based on the layout structureof which they are a child.

[0665] The Top Level Structure works as a default layout structure forleaf nodes that have no relevant layout rules specified in theMasterTable. The Top Level Structure is special in that it is the onlystructure that may itself directly contain a child layout structure.

[0666] Structures are layout structures that are children of the TopLevel Structure, whereas sub-structures are Layout Structures that areused to group leaf nodes that are located-in or attached-to other leafnodes. Note that the concept of attachment and containment are the causeof the arbitrary depth of the layout hierarchy tree. Intuitivelyarbitrary objects may be attached to one another and similarly objectsmay be placed inside of other objects to, effectively, arbitrary depth.

[0667] The grouping objects are akin to the C3DGroup objects and theleaf nodes are akin to the C3DLeaf objects of the Scene Graph. Theparent-child relationships (edges of the layout hierarchy) are mirroredin structure in the Scene Graph, thus the locations of children may bespecified relative to the parent object. For more detailed design ofC3D.

[0668] 4.1.2.3 Processing

[0669] This section provides a detailed description of the processingthat occurs in the phases of building the Geo View universe outlined inSection 4.1.1.4 of this part. Description of the individual classmethods is included in Section 5 of this part.

[0670] 1. Root Initialisation

[0671] The singleton instances of LltEdgeSet and LH_CoordGroupSet arecreated during the initialisation of the Root (LitRoot) of the layouthierarchy The Top Level Structure of the hierarchy is also created,which acts as the default structure node for leaf nodes without parentstructures. The root is itself created during the GeoViewinitialisation. Each of the singletons are children of Root, and theRoot itself is inserted into the Scene Graph, effectively becoming theScene Graph parent of the layout hierarchy

[0672] 2. Event Insertion and Removal

[0673] In this phase, entire objects or attributes of objects are addedor removed from the layout hierarchy data structure. The four operationsare ADELOBJECT, ADD_ATTRIBUTES, REPLACE OBJECT and REPLACE_ATTRIBUTES.This is the initial phase and is instigated by the insertion of eventsby World Monitor.

[0674] Object attributes may be inversable. This means that theattribute relates to two objects, and can be read in a left to right orright to left manner. For example, for Object A and Object B, if we haveA is_contained_within B, then the inverse of this is B contains A. Whensuch attributes are added to an object the inverse of the relationshipis added to the secondary object. If attributes are removed, then suchinversed attributes must also be removed from the secondary objects.

[0675] Attributes are added to, or removed from, objects that correspondto leaves in the layout hierarchy class structure and which correspondto graphical objects in the GeoView universe. These leaf nodes are thencached for further processing at the end of the insertion of a batch ofobjects and attributes.

[0676] Section 5 of this part contains method descriptions for thefollowing cases:

[0677] Insertion;

[0678] Adding Inverse Attributes;

[0679] NetworkObject Insertion and Removal; and

[0680] Attribute Insertion

[0681] 3. Leaf Building

[0682] In this phase, graphical objects (leaves) are built according totheir visual mappings in the MasterTable.

[0683] The LH_Leaf class has an aggregation relationship (‘contains’) tothe CV_(—)3DObject class, which in turn is derived from theGeneric3DObject class. The 3D object class is a descendant of the C3Dobject classes (see reference [6]) that implement rendering. When visualinformation about a Layout Hierarchy object (leaf node) arrives viaevents it is passed on to the respective CVjDObject instance.

[0684] 4. Layout Rule Application

[0685] In this phase, the parent-child relationships of leaf nodes tostructure nodes and each other are made on the basis of rules in theMasterTable. The structure nodes that are required to perform placementof leaf nodes is cached, i.e. data structures detailing the hierarchicalrelationship of objects is built, but not physically yet layed out.

[0686] Section 5 of this part contains method descriptions for thefollowing cases:

[0687] Apply Layout Structure Rules to Objects;

[0688] Handle Generic Layout Structures;

[0689] Handle Instance Matches;

[0690] Apply Attached-to Rules to Objects;

[0691] Apply Located-in Rules to Objects;

[0692] Find Satisfied Rules; and

[0693] Compose Objects.

[0694] 5. Leaf Edge Creation

[0695] In this phase, all edges in the GeoView Universe are placed inthe LH_EdgeSet singleton when they are created. The location of theirendpoint's are not maintained in the traditional hierarchical relativefashion via transform groups in the Scene Graph since they do notconceptually have a single location, but span two. As a result of this,each time a node or subtree in the layout hierarchy tree changes, eachedge must have its position updated.

[0696] Section 5 of this part contains method descriptions for thefollowing cases:

[0697] Create Edges;

[0698] Create Edge;

[0699] Update Edges;

[0700] Add Edge; and

[0701] Add Graph Edge.

[0702] 6. Object Relative Placement

[0703] In this phase, each level of the layout hierarchy tree undergoesplacement by caching objects requiring placement into the level above,until the root of the layout hierarchy tree is reached. In this way, wecan ensure that relative placement is called a maximum of once only oneach structure in the entire layout hierarchy.

[0704] The parent layout structure of an object places it according tothe placement algorithm of that structure. For example, a ring layoutstructure places its child objects spaced evenly around thecircumference of a circle. Each layout structure has settings particularto its layout shape.

[0705] 4.1.3 Layout Structure Template Library (LSTL)

[0706] This section describes the Layout Structure Template Library(LSTL), which consists of a set of C++ formatting classes. A formattingclass is one which contains pointers to other objects (T) and performssome kind of formatting on those objects. The LSTL classes areresponsible for placing the objects they are given into layoutstructures such as rings, stars, lines etc. The LSTL is a genericlibrary and all components of the LSTL are templates.

[0707] The LSTL contains the following template classes:

[0708] GenericGraph<T>

[0709] GenericLine<T>

[0710] GenericMatrix<T>

[0711] GenericRing<T>

[0712] GenericStar.cT>

[0713] GenericRectangle<T>

[0714] GenericTree′<T>

[0715] Each of these classes is a template, and can be instantiated tocontain any type of object. The only restriction on the object is thatit must supply the required interface functions as described in Section5. Each of the classes in the LSTL described in the sub-sections belowdefines an interface as described in Section 5 of this part.

[0716] 4.1.3.1 GenericGraph

[0717] This layout structure places the objects in a graph, based onedge connections between those objects. The graph algorithm attempts tosituate objects by spatially representing their interconnections whilstminimising edge crossovers. The user can specify the following settings:

[0718] NODE SEPARATION FACTOR: This value indicates the amount ofseparation between nodes in the graph (horizontal spacing) relative to aunit value of 1.0.

[0719] RANK SEPARATION FACTOR: As similar to node separation factor thisvalue represents the separation between ranks (vertical spacing).

[0720] ORIENTATION: This value determines whether the graph isorientated top-to-bottom or left-to-right.

[0721] The default node separation factor to rank separation factor is aratio of 1:3. FIG. 19 shows the Object Layout Structure of how the abovesettings relate to the graphs produced.

[0722] 4.1.3.2 GenericLine

[0723] This layout structure places the objects in a line. The user canspecify the following arguments:

[0724] AXIS: This determines to which axis (x, y or z) the line will beparallel.

[0725] LINEAR DIRECTION: This determines whether the line extends alongthe axis in a positive or negative direction.

[0726] ORIGIN: This determines whether the origin is located at thefront back or centre of the line.

[0727] SEPARATION: This is the amount of spacing the algorithm leavesbetween each object in the line.

[0728]FIG. 20 shows what the DIRECTION and ORIGIN line will look likewith various GenericLine combinations

[0729] 4.1.3.3 GenericMatrix

[0730] This layout structure places objects in a matrix. By defaultobjects are added into the matrix in a clockwise spiral as shown below:24 9 10 11 12 23 8 1 2 13 21 6 5 4 15 20 19 18 17 16 22 7 0 3 14

[0731] The user can specify the following arguments:

[0732] WIDTH_SEPARATION: This is the amount of space in the X axis thatis left between objects in the matrix.

[0733] DEPTH_SEPARATION: This is the amount of space in the Z axis thatis left between objects in the matrix.

[0734] DELETE_POLICY: This determines what the algorithm will do when anobject is removed from the matrix. It can either leave a gap, fill inthe gap with the last object or shuffle back all of the objects afterthe gap.

[0735] ORIGIN_POLICY: Determines where the true centre of the matrix islocated, either where the first object in the matrix is placed or at thetrue centre.

[0736] 4.1.3.4 GenericRing

[0737] This layout structure places objects in a ring. The user canspecify the following arguments:

[0738] ANGULAR DIRECTION: This determines the direction in which objectsare placed on the ring. It can be either clockwise or anti-clockwise.

[0739] RADIUS: This is a minimum radius for the ring. The algorithm willdetermine a dynamic radius based on object size and separation and if itis less than the user specified radius it will not be used. If it isgreater it is used rather than the user specified one.

[0740] SEPARATION: The amount of separation to leave between objects.The greater the separation the greater the dynamic radius of theresulting ring.

[0741]FIG. 21 shows a five-object ring with CLOCKWISE direction. Theorigin will always be at the centre of the ring. If a ring contains onlyone object then it will be placed at the origin.

[0742] 4.1.3.5 GenericStar

[0743] This layout structure places objects in a star One object will beassigned by the user as the root of the star and placed at the origin.The rest of the objects will be the leaves of the star and will beplaced using a GenericRing. As well as the GenericRing arguments theuser can also specify:

[0744] ROOT_HEIGHT: This is the amount that the root of the star israised above the plane.

[0745] 4.1.3.6 GenericRectangle

[0746] This layout structure places objects in a rectangle. The user canspecify the following arguments:

[0747] ANGULAR DIRECTION: The direction (clockwise or anti-clockwise) inwhich objects are placed around the rectangle.

[0748] START_SIDE: The side on which to start layout. The sides arenumbered 0-3 with 0 being the top (far) side and subsequent sidesextending clockwise.

[0749] WIDTH_SEPARATION: The separation between objects in the widthaxis.

[0750] DEPTH_SEPARATION: The separation between objects in the depthaxis

[0751] WIDTH: Specifies the width dimension of the resulting rectangle.

[0752] DEPTH: This specifies the actual dimensions of the resultingrectangle.

[0753] If width or depth values are provided, then the radius of theobjects, WIDTH_SEPARATION and DEPTH_SEPARATION will not be used in thelayout.

[0754] 4.1.3.7 GenericTree

[0755] This layout structure, like GenericGraph, also places the objectsin a Graph, based on edge connections between those objects. GenericTreeuses the same graph algorithm to determine the layout, but withdifferent parameters. The ‘Tree graph is a directed edge graph, whereedge direction is determined by the MasterTable's layout rules. Forexample, if the MasterTable specifies a Shared_Data_Link's layout ruleas:

[0756] “layout-structure Tree is_connected_to==type_of(Computer)”,

[0757] any Shared Data Link network object connected to a Computer, willbe laid out as a Tree, with the direction of the edge from the SharedData Link to the Computer. In this way, rather than the layout of a Treebeing non-deterministic given the same set of events, the Tree will belaid out in the same way each time. However, there are a few exceptionsto this rule. If two objects of the same type are connected, or if atleast one of the nodes is a structure node, then the direction becomesnon-deterministic, like GenericGraph.

[0758] 4.2 Concept of Execution

[0759] The execution concept for the GeoView module, including the flowof control, is described in Section 4.1 and Section 4.3 of this part.

[0760] 4.3 Interface Design

[0761] Like DataView, GeoView interfaces with the Registry module andTardis, which allow events and commands to be sent to it. The eventsarrive from the Intelligent Agents, and the commands arrive from theuse; via different user tools, such as the Navigation System (nay) andthe Configuration Editor (ConfigEd).

[0762] The Tardis handles incoming events from the agents, and commandsare sent to GeoView via the Command Control Interface (CCI).

[0763]FIG. 22 shows the process interactions between the ViewApplications (GeoView/DataView), the Registry, and the Tardis.

[0764] 5. Module Detailed Design

[0765] This section contains, for the GeoView module, the detaileddesign descriptions for the GeoView, LayoutHierarchy andLayoutStructureTemplateLibrary classes.

[0766] 5.1 Geo View Classes

[0767] 5.1.1 Geo View Class Summary

[0768] Table 3 identifies a full list and description of the GeoViewclasses. TABLE 3 GeoView Classes Class NameDescription Description GVAction The Geo View specific action class. GV ActionMan The GeoViewspecific action manager class. Knows how to iterate over LayoutHierarchyto refuter all nodes. GV 300bject The GeoView 3DObject.GVPacketMotionEffect Class that allows simple motion effect.GV_EventFileReader The GeoView specific event file reader. GVWorldMonitor The GeoView specific class for World Monitor. GeoView Themain (singleton) class for GeoView applicationControlGeoViewWorldService Class for adding GeoView specific services tothe CVSVWorld CCI inter- face. GeoViewCanvas Display/interface handlingfor GeoView. GeoViewSettings Subclass of ApplicationSettings to hold theapplication specific settings for GeoView.

[0769] 5.2 LayoutHierarchy Classes

[0770] 5.2.1 LayoutHierarchy Class Summary

[0771] Table 4 identifies a full list and description of theLayoutHierarchy classes. TABLE 4 Layout Hierarch Classes Class NameDescription LH_CoordGroup This class contains a group of nodes whose (x,z) coordinates all fall within a specific range. LH_CoordOroupSet Thisclass contains a set of related LH CoordCroup nodes. LHEdge This classstores information about one edge in the Layout- Hierarchy LWEdgeSetThis class is a container for all edges in the LayoutHierarchy LH_GraphThis class is a container for a group of nodes that are arranged as anadirected graph. LH_Leaf The lowest node in the LayoutHierarchy. AnLU_Leaf node contains a NetworkObject and a corresponding SV9DObjectwhich contains the 3D data that represents the NetworkObject accordingto the mles in the MasterTable LU_Line This class is a container for agroup of nodes that are arranged in a line. LH_Matrix This classrepresents a Matrix layout structure. Nodes are added to the Matrix in aclockwise spiral. LH_Node This is the base class for all nodes in aLayoutHierarchy. This class maintains the following variables LH_Root*mRootOfLfl a pointer to the root of the LayoutHierarchy. LH_Node*mParent a pointer to the parent of this node C3DBranchflroup*mBranchGroup a pointer to the branch group that contains the geometry ofthis node. This branch- group is attached to the branchgroup of thisnodes parent. This means that mRootOfLH mBranchOroup contains thegeometry for the entire LayoutHierarchy (via a bit pattern). mt mPlacedthis variable contains information about what layout rules have beenused to place this node LH_Ring This class represents a ring shapedlayout structure. A ring has a LU_Leaf as its root and a list ofLU_Nodes as its children. The branchgroup of the root in placed underthis nodes' branchgroup and the branchgroup of the children are placedunder the roots' branchgroup LU_Root This class is the top level node inthe LayoutHierarchy. It is responsible for maintaining a list of otherLH_Node objects and performing layout operations on them. It containspointer to the MasterTable that is used for layout LU_SNode This classis the base class for all structure nodes in the LayoutHierarchy. Allstructure classes (LU_Star LU_Matrix LU_Line etc) inherit from this baseclass and it provides an extra interface on top of the standardLI-I_Node interface LU_Star This class represents a star shaped layoutstructure. A star has a LU_Leaf as its root and a list of LH Nodes asits children The branchgroup of the root in placed under this nodes'branchgroup and the branchgroup of the children are placed under theroots branchgroup LU⁻Tree This class is a container for a group of nodesthat are arranged as a directed graph, where edge direction isdetermined by layout relationships between different objectsNetworkObject Contains information about a NetworkObject

[0772] 5.2.2 Event Insertion and Removal Methods

[0773] 5.2.2.1 cfInsert

[0774] This is the insertion method called directly by the World MonitorA command, a network object and an address are specified. Dependant onthe command objects being added or removed, attributes are added orremoved from the GeoView world. LH_Root::cflnsert( COMMAND,NetworkObject) On the basis of the COMMAND:- (where COMMAND = ADD OBJECTor ADD_AflRIBUTES) addInverseAttributes( networkobject ) [Section5.2.2.2] cflnsertNetworkObjectAdd( networkobject ) [Section 5.2.2.3](where COMMAND = REPLACE_OBJECT) removelnverseAttributes ( networkObjectcflnsertNetworkObjectReplace( networkobject ); [Section 5.2.2.3]addInverseAttributes ( networkObject ); [Section 5.2.2.2] (where COMMAND= REPLACE_ATITRI13UTES) removeInverseAttributes (networkObect)cfInsertNetworkObjectAttributesReplace ( networkObject) [Section5.2.2.3] addInverseAttributes( networkObject ) [Section 5.2.2.2]

[0775] 5.2.2.2 Adding Inverse Attributes

[0776] Each attribute is checked to see if it has a correspondinginverse attribute. If so a lookup of the secondary object is made. If itdoes not exist it is created and the inverse attribute is added to it,otherwise the inverse attribute is added to the existing secondaryobject.

[0777] LH_Root::addInverseAttributes( networkObject) FOR each attributein the networkobject IF there exists an inverse relationship Find theobjectname from the value of the attribute IF the leaf named objectnamedoes NOT exist   Create a leaf named objectname ENDIF Add the inverserelationship to the leaf using the name of the object   ENDIF END FOR

[0778] 5.2.2.3 NetworkObject Insertion

[0779] Each of the attributes from the passed in network object areadded to the network object of the leaf node. The leaf node is thencached for further processing after the entire current batch of eventshas arrived.

[0780] LH_Root::cfInsertNetworkObjectAdd(networkObject, address) FOReach attribute in the networkobject Call cfInsertAttrib( attribute,address ) [Section 5.2.2.4] END FOR Call layout(leaf)

[0781] 5.2.2.4 Attribute Insertion

[0782] If the leaf object specified by address does not yet exist then anew leaf is created and added to the lookup hash map. Otherwise theattribute is addded to the existing leaf.

[0783] LH_::cflnsertAttrib(attribut, address) Do a lookup of leaf usingaddress IF the leaf does not exist yet Create the leaf Add attribute toleaf Add leaf to leaf hash map OTHERWISE Add attribute to leaf IFattribute added successfully IFattribute added was LOCATED_AT Do anynecessary location processing ENDIF ENDIF ENDIF

[0784] 5.2.3 Layout Rule Application Methods

[0785] 5.2.3.1 applyLayoutstructureRulesToObject

[0786] Structure Rules specify the logical groupings of world objects(represented as leaf nodes in the virtual world) by mapping them tolayout structures on the basis of attribute tests. If a relevant layoutstructure is found via the structure rules then it is placed into thisstructure (which is created if necessary).LH_Root::applybayoutStructureRulesToObject( leaf) call findSatisfiedRules(  “layout-structure”,  rules, satisfiers ) on theleaf [Section 5.2.3.4] IF a satisfying layout structure was found Process depending on the type of the layout structureLAYOUT_STRUCTURE_LINE IF the leaf is not already in a line layoutstructure Call handleGenericLayoutStructure ( rule, satisfiers, leaf,LINE)  END IF   LAYOUT_STRUCTURE_RING    IF the leaf is not already in aring layout structure  Call handleGenericLayoutStructure (rule,satisfiers, leaf, RING)    ENDIF   LAYOUT_STRUCTURE_MATRIX    IF theleaf is not already in a matrix layout structure  CallhandleGenericLayoutStructure C rule, satisfiers, leaf, MATRIX    ENDIF  LAYOUT_STRUCTURE_STAR   The star layout structure is speciallyhandled.   handleStarLayoutStructure ( )   LAYOUT_STRUCTURE_GRAPH   Notethat since graph is designed to merge with other   structure types, nocheck of being in an existing   graph is made here.    CallhandleGenericLayoutStructure( rule, satisfiers, leaf, GRAPH) ENDIF

[0787] 1. handleGenericLayoutStructure

[0788] Assign the primary leaf node (the ‘this’ object) and thesecondary leaf (parameter ‘leaf’) to an appropriate structure (whichwill possibly need to be created) on the basis of each satisfyingattribute.

[0789] LH_Root::handleGenericLayoutstructure( rule, satisfiers, leaf,structure) ITERATE over each of the satisfier attributes IF there existsa secondary leaf (ie. This is a two-way relationship match) test if anyprimary leaf ancestor is in type of structure test if any secondary leafancestor is in type of structure IF neither are in a structure already Create a structure of type structure and add them both to it OTHERWISEIF both in different structures  merge those two structures into one oftype structure OTHERWISE one is not in a structure  add it to the onethat IS ENDIF OTHERWISE handle an Instance Match ENDIF (there exists asecondary leaf node) END ITERATION (over each attribute)

[0790] 2. handleinstanceMatch

[0791] Each instance structure is stored using a unique objectTypeName:mappingLayoutRule key. For each instance it is checked to see if astructure for this particular layout rule and type already exists; if soit is added to it, otherwise an entirely new structure with thisobject's type and rule signature is created.LH_Root::handlelnstanceMatch( Structur , node, rule, rootOfStar) Createa unique object key using CfItem: :makeAddress with the object type andthe rule name IF a structure currently has this signature add this leaf  node to that structure  OTHERWISE   create the new structure   add thestructure with unique key to a lookup hash map   add this leaf node tothe structure  ENDIF

[0792] 5.2.3.2 applyAttachedToRulesToOb]ect

[0793] Placement Rules specify the attachment and containmentrelationships of world objects on the basis of attribute tests. If arelevant attachment relationship is found via the layout rules then theprimary object is either placed into an attachment relationship as theparent (i.e. things are attached to it) or as the child (i.e. attachedto something). During this process any relevant layout rule arguments(LRAs) are read. LH_Root::applyAttachedToRulesToObject( leaf) CallfindSatisfiedRules( “attached-to”, rules, satisfiers ) on theleaf[Section 5.2.3.4] IF any satisfying rules were found   ITERATEthrough each satisfying rule found    read any layout rule arguments forthis layout rule    IF this is NOT an inverse rule (The primary objectis attached to a single other parent) find secondary leaf node using theattribute value and the leaf hash map set sub-structure scaling on thebasis of any layout rule arguments Call composeObjects( primary leaf,secondary leaf, “attached-to”, LRA's) OTHERWISE (The primary object isthe parent of the attached-to relationship) ITERATE through eachsatisfying attribute found  find the secondary leaf node via theattribute value and lookup  set sub-structure scaling on the basis ofany layout rule arguments  Call composeObjects( second leaf, prime leaf,“attached-to”, LRA's) END ITERATION (each attribute)    END IF (rule isinverse)   END ITERATION (each rule) END IF (any satisfying rules found)

[0794] 5.2.3.3 applyLocatedinRulesTo Object

[0795] Placement Rules specify the attachment and containmentrelationships of world objects on the basis of attribute tests. If arelevant containment relationship is found via the layout rules then theprimary object is either placed into a containment relationship as theparent (i.e. things are contained within it) or as the child (i.e.inside of something). During this process any relevant LRAs are read.

[0796] LH_Root::applyLocatedlnRulesToObject( leaf) Call  findSatisfiedRules(   “located-in”,   rules, satisfiers on theleaf)[Section 5.2.3.4] IF any satisfying rules were found  ITERATEthrough each satisfying rule  Read any layout rule arguments associatedwith the  rule  IF the rule is NOT inversed  (Primary leaf node will belocated in another leaf  node)  find the secondary leaf node via lookupusing the  first satisfier attribute value  call composeObjects( primaryleaf, secondary leaf,  “located-in”, LRAs)  OTHERWISE  (Secondary leafnode will have other leaf nodes located within it)  ITERATE through eachof the satisfying attributes   find the current secondary leaf node vialookup   with attribute's value  call composeObjects(secondary leaf,primary leaf,  “located-in”, LRA)  END ITERATION (each satisfyingattribute)  END IF (rule is inverse?) END ITERATION (each satisfyingrule)  END IF (any satisfying rules were found)

[0797] 5.2.3.4 findSatisfledRules

[0798] Find any matching structure rules from the MasterTable for theleaf node. For any found, record the rule matched, and an array of thesatisfying attributes. Wildcards may be matched if they are present inthe MasterTable. Processing of the unique LRA is done in this functionalso, matching instances as necessary.LH_Node::findSatisfiedRules(ruleType,  returned  list  of  matchingrules, returned array indexed by matched rules of a list of attributesthat match the rule)  get the networkObject for this leaf node build thelist of  layout mappings associated with objects of this type via cfGetChildren in mTable (eg. “MasterTable:GeoView:Computer:layout-structure”) append  to this listany WILDCARD matches  ITERATE through each mapping layout  get theObjectAttributeTest for the napping layout  IF the layout rule from themapping layout is of the required rule type   IF the secondary object isa WILDCARD    (Secondary object wildcard processing)    create any rulesand satisfying attributes for this wildcard   OTHERWISE (secondaryobject is not a WILDCARD) (Secondary object normal processing)   IF theright hand side of the layout rule represents an object type   (Relationship processing)    call getAttributesThatSatisfy to buildsatisfying attributes on OAT    OTHERWISE    (Do instance processing)   IF there is a unique flag in the Layout Rule Arguments     calldoUniqueLRAProcessing( TODO )    OTHERWISE (no unique flag)     findfirst attribute    END IF (unique flag exists?)

[0799] 5.2.3.5 composeObjects

[0800] The passed in parent and child objects are composed or aggregatedinto an object composition via attachment or containment, i.e. withcontainment the child is contained within the parent and with attachmentthe child is attached to the parent. A child cannot be a descendant ofparent is asserted.

[0801] Special processing is done in the case where the child is in alayout structure already and the parent is not. In this case the childis removed from the layout structure, composed with the parent, and thenthe entire object composition is re-inserted back into the originalchild layout structure.

[0802] Consider the case where both the parent and child are already inlayout structures. In this instance the parent takes precedence and assuch the child is removed from its layout structure and composed withthe parent (implicitly placing it into the parent's layout structure.)

[0803] LH_Root::composeObjects( parent, child, composition type) IF thechild is already attached-to or located-in     EXIT function END IF(child already attached-to or located-in) IF the child is an ancestor ofthe parent     REPORT error END IF (check not ancestor) SET childstructure to the layout structure (if any) that the child is in IFcomposition type is attachment   Call attach( child ) on parentOTHERWISE (composition type not attachment)   Call contain( child ) onparent END IF (composition type) IF the child WAS in a layout structureAND the parent is not   INSERT the new composite object into the childstructure END IF (child was in layout structure and parent isn't)

[0804] 5.2.4 Leaf Edge Creation Methods

[0805] 5.2.4.1 createEdges

[0806] Create any new edges that are to be associated with this leaf.During this processing, non visible edges are updated for LSTLcomponents (for example Graph and Tree structures.)LH_Leaf::createEdges( ) ITERATE through attributes associated with thisleaf IF  the attribute's name is “IS_CONNECTED_TO”   Call createEdge(attribute ) [Section 5.2.4.2]  END IF (name is connected to) ENDITERATION

[0807] 5.2.4.2 createEdge

[0808] Using the attribute, find the connected-to node. Ensuring thereis no current visible edge, create a new one to it. LH_Leaf::createEdge(matching attribute) SET connectedToNode by using the string value of thepassed in attribute Call cfGetReference( connectedToNode ) to find thenode's leaf instance (if any) IF the node is found AND there is nocurrent connection to it   SET absloc to the absolute location of thecurrent node   SET conabsloc to the absolute location of the connectedTonode   Call addEdge( this node, bc, conn. node, conloc) on edgeSetsingleton [Section 5.2.4.4]   Call addEdge( edge ) to add any nonvisible graph edges to this node[Section 5.2.4.51 END IF (node found andno current connection)

[0809] 5.2.4.3 updateEdges

[0810] Update edge locations on the basis of this Leaf's location.LU_Leaf::updateEdges( ) IF the delay processing flag is set   cache thecurrent leaf for edge processing later   END IF (delay processing flag)ITERATE through each of mEdges   Call setLocation( ) on the edge andmake it the absolute location of this leaf END ITERATION IF there is anattached-to structure node   Call updateEdges( ) on the structure nodeEND IF (attached-to structure node) IF there is a located-in structurenode   Call updateEdges( ) on the structure node END IF (located-instructure node)

[0811] 5.2.4.4 addEdge

[0812] An edge is added to the EdgeSet singleton. LRYdgeSet::addEdge(node1, location1, node2, location2) IF currentline modulus 100 yields noremainder ALLOCATE space for another 100 lines and set them END IF(modulus 100) Create a new edge Add it to mEdges Check for edgevisibility and add it to the appropriate list

[0813] 5.2.4.5 addGEdge

[0814] A non-visible edge interconnection is added on the basis ofwhether there is a common Graph (or graph sub-typed) parent. This keepsedge information for Graph and its descendents in the LSTL up to date.

[0815] LH_Leaf::addGEdge( Edge)

[0816] SET childNode1 via calling getNode1( ) on edge

[0817] SET childNode2 via calling getNode2( ) on edge

[0818] Look for a common graph/tree via calling findCommonSNode( ) onLH_SNode

[0819] Add structure edge to the leaf

[0820] 5.3 LayoutStnictureTemplateLibrary (LSTL) Classes

[0821] 5.3.1 LSTL Template Class Summary

[0822] Table 5 identifies a full list and description of the LSTLclasses. TABLE 5 LSTL Classes Class name Description GenericGraph Thistemplate class places the objects in a graph. GenericLine This templateclass places the objects in a line. GenericMatrix This template classplaces objects in a matrix. GenericRing This template class placesobjects in a ring. GenericStar This template class places objects in astar GenericRectangle This template class places objects in a rectangle.GenericTree This tem late class laces oWeds in a tree.

[0823] 5.3.2 GenericGraph Methods

[0824] 5.3.2.1 Node Separation Factor

[0825] This value indicates the amount of separation between nodes inthe graph (horizontal spacing) relative to a unit value of 1.0. Values:Positive floating point Interface: float getNodeSeparationFactor() constvoid setNodeSeparationFactor( const float val)

[0826] 5.32.2 Rank Separation Factor

[0827] As similar to node separation factor this value represents theseparation between ranks (vertical spacing). Values: Positive floatingpoint Interface: float getNodeSeparationFactor() const voidsetNodeSeparationFactor( const float val)

[0828] 5.3.2.3 Orientation

[0829] This value determines whether the graph is orientatedtop-to-bottom or left-to-right. Values: TOP_TO_BOTTOM, LEFT_TO_RIGHTInterface: Graph Orientation Policy getOrientation() const voidsetOrientation( const Graph_Orientation_Policy orient)

[0830] 5.3.3 GenericLine Methods

[0831] 5.3.3.1 Axis

[0832] This determines which axis (x, y or z) the line will be parallelto. Values: X_AXIS, Y_AXIS, Z_AXIS Interface LSTL_LineAxis getAxi()const void setAxis( const LSTL_LineAxis axis)

[0833] 5.3.3.2 Linear Direction

[0834] This determines whether the line extends along the axis in apositive or negative direction. Values: POSITIVE, NEGATIVE Interface:LSTL_LinearDirection_getDirection( ) const void setDirection( constLSTL_LinearDirection dir)

[0835] 5.3.3.3 Origin

[0836] This determines whether the origin is located at the front backor centre of the line. Values: FIRST. LAST, CENTER Interface:LSTL_LineOrigin getOrigi( ) const void setOrigin( const LSTL_LineOriginorigin)

[0837] 5.32.4 Separation

[0838] This is the amount of spacing the algorithm leaves between eachobject in the line. Values: Positive floating point Interface: floatgetSeparatio( ) const void setSeparation( const float sep)

[0839] 5.3.4 GenericMatrix Methods

[0840] 5.3.4.1 Width Separation

[0841] This is the amount of space in the X axis that is left betweenobjects in the matrix. Values: Positive floating point Interface: floatWidthSeparation( ) const void WidthSeparation( const float sep)

[0842] 5.3.4.2 Depth Separation

[0843] This is the amount of space in the Z axis that is left betweenobjects in the matrix. Values: Positive floating point Interface: floatDepthSeparation( ) const void DepthSeparation( const float sep)

[0844] 5.3.4.3 Delete Policy

[0845] This determines what the algorithm will do when an object isremoved from the matrix. It can either leave a gap, fill in the gap withthe last object or shuffle back all of the objects after the gap.Values: LEAVE GAP FILL_GAP_FROM_END, SHUFFLE Interface:LSTL_deletePolicy_getDeletePolic( )_const void setDeletePolicy( constLSTL_deletePolicy policy)

[0846] 5.3.4.4 Origin Policy

[0847] Determines where the true centre of the matrix is located, eitherwhere the first object in the matrix is placed or the true centre.Values: FIRST, CENTER Interface: LSTL OriginPolicy getOriginPolicy( )const void setOriginPolicy( const LSTL_OriginPolicy policy)

[0848] 5.3.5 GenericRing Methods

[0849] 5.35.1 Angular Direction

[0850] This determines the direction in which objects are placed on thering. It can be either clockwise or anti-clockwise Values: CLOCKWISE,ANTI-CLOCKWISE Interface: LSTL AngularDirection getDirection( ) coustvoid setDirection( const LSTVAngularDirection dir)

[0851] 5.3.5.2 Radius

[0852] This is a minimum radius for the ring. The algorithm willdetermine a dynamic radius based on object size and separation and if itis less than the user specified radius it will not be used. If it isgreater it is used rather than the user specified one. Values: Positivefloating point Interface: float getRadius( ) coost void setRadius( constfloat radius)

[0853] 5.3.5.3 Separation

[0854] The amount of separation to leave between objects. The greaterthe separation the greater the dynamic radius of the resulting ring.Values: Positive floating point Interface: float getNodeSeparation( )const void setNodeSeparation( const float nodeSeparation)

[0855] 5.3.6 GenericStar Methods 52.6.1 Root Height

[0856] This is the amount that the root of the star is raised above theplane. Values: Positive floating point Interface: float getRootHeightOconst void setRootHeight( float rootHeight)

[0857] 5.3.7 GenericRectangle Methods

[0858] 5.3.7.1 Angular Direction

[0859] The direction (clockwise or anti-clockwise) in which objects areplaced around the rectangle. Values: CLOCKWISE, ANTI-CLOCKWISEInterface: LSTL AngularDirection getDirection˜const void setDirection(const LSTLAngularDirection ang)

[0860] 5.3.7.2 Start Side

[0861] The side on which to start layout. The sides are numbered 0-3with 0 being the top (far) side and subsequent sides extendingclockwise. Values: Integral range [0..3] Interface: int getStartSide( )const void setStartSide( int startSide)

[0862] 5.3.7.3 Width Separation

[0863] The separation between objects in the width axis. Values:Positive floating point Interface: float getWidthSeparatin( ) const voidsetWidthSeparation( const float widthSeparation)

[0864] 5.3.7.4 Depth Separation

[0865] The separation between objects in the depth axis. Values:Positive floating point Interface: float getDepthSeparation( ) constvoid setDepthSeparation( const float depthseparation)

[0866] 5.3.7.5 Width

[0867] Specifies the width dimension of the resulting rectangle. Values:Positive floating point Interface: float getWidth( ) const voidsetWidth( const float width)

[0868] 5.3.7.6 Depth

[0869] This specifies the actual dimensions of the resulting rectangle.Values: Positive floating point Interface: float getDepth( ) const voidsetDepth( const float depth)

[0870] 5.3.8 LSTL Class Interface

[0871] Each of the classes in the LSTL defines a common interface asshown in Table 6. TABLE 6 LSTL Class Interface Method Desciptioniterator getFirst ( ) const Get art iterator to the first object in Getan iterator to the first object the structure. in the structure. NOTE:The type of iterator is defined in the class itself Currently it isvector <T*>. Use GenericStructure<Foo>::iterator as the type may change.iterator getLast( ) const Get an iterator to the last object in thestructure const_iterator getFirstConst( ) Return constant iterator tobeginning const of children. const_iterator getLastconst( ) Returnconstant iterator to end of const children. int getNumChildren( ) constGet the number of objects in the structure. void insert( T*, element)Insert the given object into the structure. Layout will be called ifdoLayout is true (This is the default). void relativePlac ment( )Perform layout on the objects in the structure. Perform layout on theobjects in the structure. void remove( T*, element) Remove an objectfrom the structure. Remove an object from the Layout will be called ifdoLayout is structure. Layout will be called if true. (This is thedefault) doLayout is true. (This is the default) voidset<ATTRIBUTE>(arg) Set the appropriate attribute. Get<ATTRIBUTE> ( )Get the appropriate attribute.

[0872] Each structure may have additional methods that only apply to it.More details can be found by looking at the interface of a particularclass in automatically generated documentation or the header files.

[0873] 5.3.8.1 Memory Allocation

[0874] The template classes are not responsible for memoryallocation/de-allocation for the T * objects. In the users applicationthe T objects should be maintained and pointers passed to the structuretemplates. The application will be responsible for complete control ofthe T objects.

[0875] 5.3.8.2 Relative Placement

[0876] It is up to the user of the template object instance to callrelativePlacement( ) when they want the layout algorithm to run for aparticular layout structure. The layout algorithms will use thetemplated objects’ getBoundsRadius( )call to ensure no overlap of theobjects that are being placed.

[0877] 5.3.9 T Interface

[0878] The object for instantiating a LSTL class must provide theinterface as shown in Table 7. TABLE 7 T Interface Method Descriptionvoid setLocaeion( float x, float y. Each layout algorithm will callfloat z) this method in order to set the loca- tion for each object voidgetLocation ( float& x, float& y. The current location of the object.float& z) char* getId ( ) A unique identifier for the object. floatgetBoundsRadius( ) Each algorithm will take into account the size ofeach object in the structure when laying them out. This call shouldreturn the radius of a sphere which completely encompasses the object.

[0879] 6. Appendix for GeoView

[0880] This section contains a glossary to the SDD for the GeoViewmodule. It contains abbreviations and definitions of terms used in theSDD.

[0881] 7.1 Abbreviations

[0882] The following are abbreviations used in this document.Term/Acronym Meaning CCI Component Control Interface CSCI ComputerSoftware Configuration Item DID Data Item Description LRA Layout RuleArgument LSTL Layout Structure Template Library SDD Software DesignDescription SSDD System/Subsystem Design Description SSSSystem/Subsystem Specification

[0883] 7.2 Definition of Terms

[0884] The following are terms used in this document. Term DescriptionAddress A character string uniquely identifying the event and operation.Attachment A child object in an attached-to relationship with a parentobject. Attributes String representations of the facets of a worldobject or relationships to other world objects. Batching Grouping of twoor more events. Bounds The radius of influence about an object inGeoView. Radius Building The act of giving information to the rendererto render a leaf node. Composition Two or more objects in an attached-toor located-in relationship. Configu- The base level abstract class thatholds the name of ration an object Item and methods for insertion,deletion and lookup of other objects. Containment A child object in alocated-in relationship with a parent object. Edge A physical lineinterconnecting two leaf nodes. Events External information arriving inthe form of network objects. Layout The act of combining the processesof leaf building, layout rule application, edge creation and objectplacement. Layout Rules Rules specifying the attachment, containment orlayout structure grouping of a leaf node (representing a world object)based on its attributes. Layout A logical grouping construct that doesplacement on child Structure leaves based on the shape of the structure.Leaf Node GeoView's graphical building blocks representing objects inthe world. MasterTable A hierarchical set of mappings from world objectsto leaf nodes specifying visual attributes and layout rules. Network Ancontainer of one or more attributes. Object Node The abstract baseparent of layout hierarchy classes. Object Represents either a leaf orlayout structure in GeoView. Parent The node directly above the currentone in the layout hierarchy. Placement The act of placing an object inthe GeoView Universe either absolutely or relatively. Relationship Astring describing the logical connection between two leaf nodes. RootThe singleton instance at the top of the layout hierarchy. Scene GraphThe Java 3D API data structure for rendering in 3D worlds SingletonRecognised design pattern that is used to create a class that isguaranteed to have only one object instance in the application.Structure A Layout Structure that is a direct child of the top-levelstructure Sub-structure A Layout Structure that is a direct child of aparent leaf node. Used for grouping leaf nodes that are in a compositewith the parent node. Top Level The top most structure level of thelayout hierarchy where the parent structure is the top level structure.World Physical or logical objects that exist or are defined in theObjects real world e.g. Computer, Shared Data Link

PART 5 TARDIS SPECIFICATION

[0885] 1. Tardis Specification

[0886] Tardis is briefly discussed in Section 2.1.4 of the Shapes VectorOverview, Part 1 of this specification.

[0887] The following is a preferred specification of its characteristicsin the embodiment described. However, it is also possible for the Tardisto operate independently and/or in conjunction with other elements notrelated to elements of the preferred embodiment.

[0888] It is possible for Tardis to operate with just the Gestalt orjust one observation sub-system such as Geo View or Data View. It isalso possible to construct configurations of the Shapes Vector system inwhich the event outputs from agents is fed via the Tardis to athird-party visualisation or analysis system, or to a text-based eventdisplay. In cases where time-based queuing and semantic filtering ofevents is not required, the system could alternatively be configured insuch a way as the event outputs from agents are delivered directly toone or more of the view components in a real time visualisation system.

[0889] 1.1 Introduction

[0890] The Tardis is the event handling sub-system of Shapes Vector. Itmanages incoming events from a system Client, in a typical arrangementthe Gestalt, and makes them available for Monitors (a recipientobservation sub-system) to read. There can be many Clients and Monitorsconnected to the Tardis at the same time.

[0891] The Tardis receives events from Clients via connections throughTardis Input Portals, and uses Shared Memory as its form ofinter-process communication with Monitors. Tardis Input Portals supportdifferent types of connections, such as socket transaction.

[0892] The flow of data through the Tardis is in one direction only, theTardis reads from the connections with the Clients, and writes to SharedMemory.

[0893] 1.2 Assumptions

[0894] For the purpose of this disclosure of a preferred embodiment, itis assumed that the reader is familiar with the products, environmentsand concepts that are used with the Shapes Vector infrastructuredisclosed earlier in this specification.

[0895] 2. Overview of the Tardis

[0896] The Tardis receives events from one or more Clients/Sources thatcan be located physically dose or remote from the Tardis and suppliesthem to Recipient Systems that also can be remotely located. A Recipientsystem may also be a Client/Source. Each Client/Source associates witheach event an ordered data value that is, in an embodiment, one of anincrementing series of data values. Typically the ordered data value isrepresentative of real or synthetic time as gauged from an agreed epoch.Since the data value can be compared with other data values they areuseful for ordering events within a common queue (the term slot is alsoused in this specification to describe the function of a queue). Sincedifferent events in different queues can have the same data value theycan be identified or grouped to provide a temporal view of the eventsthat does not have to be a real time view. For example, by creating oneor more spans or changing the magnitude of the span of the data valuesoutput by the Tardis it is possible to provide control over time andthen present events to the Recipient systems relating to those times.The timed event output to a Recipient system could be in synchronisationwith real time, if desired by the user observing the system Recipientsystem output. It is also possible to change the rate of flow of thedata values selected for output from the Tardis thus controlling thetime span over which those events are presented for observation. Theremay be triggers available to initiate one or more time related outputsthat can be set by the observing user to assist their detection ofpredetermined events. Further the triggers and their effect may bedetermined by way of calculations on data values set by the user of thesystem. Not all events are of the highest importance hence there is ameans by which different priority can be allocated for each event andhandled by Tardis. So that an event's priority will determine its orderof output from Tardis and/or whether the event can be discarded undercertain circumstances such as when the system is under extreme load. Theunify bit described in this specification is an embodiment of the eventprioritization system.

[0897] There is an agreed semantic associated with each event and therewill exist in Tardis a slot for each semantic.

[0898] 2.1 Components

[0899] The Tardis uses several different threads during execution, eachfulfilling different roles within the Tardis. There is the Tardis MasterThread (M Thread), a set of Event Processing Threads (X Threads), a setof Update Threads (Y Threads), a set of New Connection Threads (ZThreads) and a set of Control Socket Threads (C Threads).

[0900] The Tardis is comprised of various data structures, such as theTardis Store, Slots, Cells, Cell Pools and their Managers.

[0901] 2.2 Overview of Operation

[0902] As the M Thread starts, it creates a set of Input Portals, whichrepresent the conduits through which Clients send events to the Tardis.Each Input Portal creates a Z Thread to manage new connections for theInput Portal. The M Thread then creates a set of X Threads (as many asspecified by the user) and a set of Y Threads (as many as specified bythe user). It also creates some C Threads for communication withexternal processes via CCI (Component Control Interface), and createsthe Tardis Store. Note that the Tardis is a process, which contains manythreads, including the original thread created by the process, the MThread.

[0903] The X Threads grab events coming in from the Input PortalConnections and place them in their corresponding queues in the TardisStore. The Tardis Store resides in shared memory. When a clock tickoccurs, an update begins, which requires the Y Threads to update thepreferred double buffered event lists (there are write lists and readlists, which switch every update, giving double buffered behaviour).When a switch occurs, a new set of event lists is presented to theMonitors.

[0904] The Tardis is able to accept a specified set ofinstructions/requests from external entities through any one of itsCCIs. This functionality is provided via the C Threads, providingexternal control and instrumentation for the Tardis.

[0905] 3. Tardis Concepts

[0906] 3.1 Events

[0907] An event is used to represent the fact that some occurrence ofsignificance has taken place within the system, and may have some dataassociated with it. There is a global allocation of event identifiers toevents with associated semantics in the system.

[0908] Conceptually, all events in the Tardis are the same, but inimplementation, there are two event formats. The first is an incoming(or network) event, as received by the Tardis via an Input PortalConnection from Clients. This event consists of an identifier, atimestamp, an auxiliary field and a variable length data field. Theauxiliary field contains the event's unify flag, type, the length of theevent's data (in bytes) and some unused space.

[0909] The second event format is an Event Cell, as used within theTardis and read by Monitors. Event Cells share some of the fields of anincoming event. They have a Cell Pool Manager pointer (which points tothe Cell Pool Manager who manages the cell), a next cell and previouscell index (to link with other Event Cells), a first Data Cell index (tolink with a Data Cell), a timestamp, an auxiliary field (same content asfor an incoming event) and a fixed size data field.

[0910] The Cell Pool Manager pointer is used when placing a cell backinto a free cell list (within the relevant Cell Pool Manager). The nextcell index is used when the cell is in a free cell list, a data Celllist or an Event Cell queue or list. The previous Event Cell index isused when the Event Cell is in an Event Cell queue. The only otherdifference between a network event and an Event Cell is that an EventCell has a fixed size data field and a first Data Cell index instead ofa variable length data field. For reasons of efficient storage, thefirst part of the variable length data field is placed in the fixed sizedata field of the Event Cell. The rest is placed in a sequence of DataCells which each point (via an index, not an address) to the next DataCell, with the last possibly being partially filled. The first of thesequence of Data Cells is pointed to by the first Data Cell index.

[0911] The identifier, auxiliary field and timestamp are 64 bits each,with the timestamp being conceptually divided into two 32 bitquantities. Within the auxiliary field, the unify flag is 1 bit, thetype is 4 bits and the data length is 16 bits (the data length isexpressed in bytes, allowing up to 64Kb of data to accompany eachevent). This leaves 43 bits of unused space in the auxiliary field.

[0912] The cell indices are all 32 bit (allowing a Cell Pool with morethan four billion cells). The size of the fixed size data field is to bespecified at compile time, but should be a multiple of 64 bits.

[0913] For strong reasons of efficiency and performance, Event Cells andData Cells are stored together in common pools and are the same size.The format of a cell (Event and Data Cell) is shown in FIG. 23.

[0914] The following are examples of some events:

[0915] 1. object information (one event id for each type of object)

[0916] signal that a new object has been discovered or that an update ofthe attributes of the object is available.

[0917] 2. object attribute information (again one event id for each typeof object)

[0918] signal that there is new or updated information for an objectattribute.

[0919] 3.2 TimeStamp

[0920] The timestamp indicates the time at which the event was generatedat the source. It consists of two 32-bit quantities indicating withsecond and nanosecond components the elapsed time since 00:00 UniversalTime (UT) Jan. 1, 1970. Note that specifying this in terms of UniversalTime allays any potential problems with events from different timezones. The timestamp is read but not modified by the Tardis. It isstored as a single 64-bit quantity, and should be stored so that theTardis using a single 64-bit instruction can compare timestamps. TheClients are responsible for ensuring the timestamp is in an appropriateformat.

[0921] 3.3 Shared Memory

[0922] The Tardis creates a shared memory segment during start-up. Thisis so that the Tardis and a number of Monitor processes have fast accessto the Tardis Store, which contains all the structures relevant to theMonitors as depicted in FIG. 24.

[0923] 3.4 Time

[0924] Dealing with time within Shapes Vector is complex and raises manyissues. The issues range from the relatively simply issue of having todeal with different time zones (from sensors distributed about theplace), to synthetic time and its relationship with events in theTardis.

[0925] 3.4.1 Universal Time

[0926] In order for events to be collated and assessed there needs to bea global clock or frame of reference for time with which events can betime encoded. The standard Universal Time (UT) is an obvious candidatefor such a frame of reference.

[0927] 3.4.2 Synthetic Time

[0928] Synthetic time is closely associated with the read lists. Theactual synthetic time indicates the time associated with the read listsas read by the Monitors.

[0929] The Tardis maintains a Synthetic Time Window, which has a width(the amount of synthetic time between the beginning and end of thewindow) and a velocity (the amount of synthetic time the window moves byafter each dock tick). The front edge (towards the future) of the windowrepresents the Current Synthetic Time. Synthetic Time and the SyntheticTime Window are shown in FIG. 25.

[0930] Updates occur at every dock tick. During the update process, theY Threads use the Synthetic Time Window to process events. Note that theSynthetic Time Window has no relation with real time, and has no bearingon the amount of real time -between updates, since the timing of anupdate is controlled by an external clock mechanism.

[0931] The Synthetic Time Window is used to guide the processing ofevents.

[0932] 3.5 Process and Thread Activity

[0933] The Monitors and Clients operate independently of the Tardis indifferent processes. The Tardis process consists of several differenttypes of Threads, whose behaviour needs to be controlled to protectshared data.

[0934] In order to control the threads, the MThread needs to be able tosignal some threads to engage and to disengage. In order to ensure athread has disengaged, the MThread needs to signal the thread todisengage, and then confirm a response from the thread indicating it hasindeed disengaged. This introduces a problem, in that the MThread maysignal a thread to disengage, but the thread in question may be busy,and will not check to see if it should disengage in a timely fashion. Inthis event, the M Thread will be wasting time waiting for the response.In some cases, this is unavoidable, however, the thread may be engagedin an activity which is thread safe. If this is the case, the MThreadshould not wait for a response from the thread, and can continue safely,so long as the busy thread checks to see if it should disengage beforeengaging in thread unsafe activity.

[0935] Hence each thread should have a flag it maintains indicatingwhether it is engaged or not. It should also have a flag it maintainsindicating whether it is safely engaged or not. Finally, the M Threadshould maintain a flag per type of thread it controls (ie. one for XThreads, one for Y Threads and one for Z Threads).

[0936] 4. Functional Overview of the Tardis

[0937] 4.1 Tardis Threads

[0938] The Tardis is made up of several different types of threads whichwork together to make the Tardis function. The M Thread is the masterthread, and controls the other threads and the update process. X Threadshave the job of reading events from the Input Portals, obtaining andpopulating Event and Data Cells and placing the Event Cells in theappropriate Slot's queue. Y Threads are called on during every update totake certain Event Cells from a Slot's queue, and to place them in theSlot's event list. Z Threads are responsible for creating newconnections with Clients through the Input Portals. C Threads areresponsible for handling CCI commands and requests.

[0939] This is shown in FIG. 26.

[0940] Note that the M Thread is the only thread that directly interactswith another thread.

[0941] The scheduling of these threads is important, and revolves aroundan update, which occurs when a dock tick occurs. When the Tardis is notdoing an update, the X Threads are handling incoming events and the ZThreads are handling new connections. When an update occurs, the X and ZThreads are disengaged and the Y Threads engaged to update the eventlists. At the end of an update, the Y threads are disengaged and the Xand Z Threads engaged again.

[0942] The M Thread and the C Threads are never disengaged.

[0943]FIG. 27 shows when each thread and process is waiting (to beengaged or for the M Thread, for a clock tick). The shaded areas showwhere the thread or process is not waiting.

[0944] The shaded areas represent time periods where:

[0945] Client processes are possibly sending events throughout the timethey are connected to the Tardis. The Tardis does not have an effect onthe process activity of Clients or Monitors. Note that a Client mayproduce a burst of events and then shutdown, or it may run for anextended period of time, possibly sending events continually orsporadically.

[0946] Monitors are able to read the current read lists. They are ableto detect any event list switching during reading. Note that if theMonitors finish their processing of the read lists and cells, they waituntil the next update to go into action again.

[0947] The Tardis is receiving events from Clients and making eventsavailable to Monitors.

[0948] The M Thread is controlling an update.

[0949] The X Threads are engaged and busy storing incoming events. Theyare also detecting Input Portal Connections that have timed out andadding them to their own “to-remove” lists of Input Portal Connections.

[0950] Y Threads are updating the next read lists (the current writelists) and discarding old non -unified events.

[0951] Z Threads are accepting Client requests for new Input PortalConnections. They are also creating new Input Portal Connections andplacing them in their own “to-add” lists of Input Portal Connections.

[0952] C Threads are servicing requests and commands received via CCI.

[0953] The X Threads loop through Input Portal Connections, and collectones which timeout, but do not modify the list of Input PortalConnections. The Z Threads create new Input Portal Connections, but alsodo not modify the list. This is to avoid X and Z Threads blocking eachother over access to the shared list. However, whilst both aredisengaged, the to-add and to-remove lists each maintained are used tomodify the shared list.

[0954] 4.2 Tardis Operation

[0955] Upon start-up, the MThread creates the shared memory segment,creates a set of Input Portals (and a Z Thread per Input Portal),creates a number of X Threads and Y Threads and then sits in a loop.When a new Client requests an input connection on an Input Portal, the ZThread for that Input Portal creates an Input Portal Connection objectwhich is later added to the M Thread's Input Portal Connection list.

[0956] The Tardis has a number of X Threads responsible for themanagement of incoming events. X Threads grab events from Input PortalConnections, so each Input Portal Connection needs to be protected by alock. These events are stored directly into the event queue of theappropriate Slot by the X Threads, so each Slot needs to be protected bya lock. Hence an X Thread can be blocked attempting to get the lock onan Input Portal Connection, and then on the resulting Slot. This shouldbe expected, and by having many X Threads, such blocking need notsignificantly affect performance (the more X Threads there are, the moreblocking will occur, but it will be less significant because other XThreads will use the time constructively).

[0957] When a clock tick occurs, the M Thread begins an update. First itflags the X Threads and Z Threads to disengage and ensures they aredisengaged or safely executing. Then it signals the Y Threads to engage.When the Y Threads have finished the update, they are disengaged and theX and Z Threads are engaged.

[0958] The MThread then updates the current synthetic time, switches theevent lists, increments the update counter and prepares the write listsfor writing (discarding events in the write lists, which have been readby Monitors). The order of the last operations is critical as thecurrent synthetic time must be updated before the event lists areswitched which must be done before incrementing the update counter. Theorder is used by the Monitors to detect a switch and preserve dataintegrity.

[0959] The Tardis uses multiple Z Threads (one per Input Portal) toaccept new Client requests for an Input Portal Connection. For thepurpose of protecting data from being written to whilst being read, orwritten to simultaneously, the Z Threads are placed in a wait state atthe same time as the X Threads, and started again at the same time asthe X Threads. This means that at any one time, either the Z Threads orthe M Thread has access to the Z Threads' to-add lists.

[0960] However, the Z Threads may be blocked whilst accepting newconnections, so the Z Threads indicate if they are in a safely executingstate. The Z Threads relieve from the MThread the job of accepting andcreating new connections, which leaves the M Thread better able tomaintain responsiveness.

[0961] The X and Y Threads may also declare themselves as safelyexecuting in order to reduce the latency that comes with waiting for allX or Y Threads to disengage.

[0962] 4.3 Tardis Store

[0963]FIG. 28 gives an overview of the array of Slots residing withinthe Tardis Store in shared memory. Each Slot has an index to the firstand last Event Cells in its Event Cell queue. It also has an index tothe first event in the read and write lists. All Event Cells and DataCells are from a Cell Pool, although which pool does not matter.

[0964] In order to store an event, X Threads first look-up the event idin a Slot Mapping Array. This returns an index to the array of Slots.The Slot contains all the entities the X Thread needs to perform itsoperations (indices, lock, Guaranteed Cell Pool, unify flag etc.). Withthis information, the X Thread can obtain and populate the Event Celland required Data Cells. The X Thread can also insert the Event Cell inthe Slot's queue after getting hold of the lock for that Slot (as therecould be multiple X Threads trying-to insert Event Cells in the sameSlot's queue). The event queue for each Slot is time-ordered (based oneach Event Cell's timestamp). The last Event Cell in the queue has thelargest timestamp the first in the queue is the smallest. The eventqueue is represented by the first and last Event Cell indices.

[0965] The event lists shown in FIG. 29 have their roles switch betweenthe read and write lists each update. These lists are represented by anindex to the first Event Cell in the list (the oldest). The lists areseparated (broken) from the queue by clearing the index pointers betweenthe newest event in the list and the oldest event in the queue. Hencethe Y Threads merely manipulate Slot and Event Cell indices.

[0966] When a switch occurs at the end of an update, the event listnominated as the write list becomes the read list (from which Monitorscan access the events) and the event list nominated as the read listbecomes the write list (which Y Threads will manipulate during the nextupdate).

[0967] The event lists are strictly controlled via several variables foreach Slot. These define:

[0968] 1. The maximum number of events allowed in an event list.

[0969] 2. The maximum number of unified events allowed in an event list.

[0970] 3. The maximum number of non-unified events allowed in an eventlist.

[0971] The variables are adhered to in the order of the potentialevents. Table 1 below gives some examples for a potential event queueof: “U, U, N, U, N”, with the last event at the head: TABLE 1 Max NonAdded Non Max Events Max Unified Unified Added Unified Unified 1 1 1 0 110 10 0 3 0 5 5 5 3 2 4 3 3 2 2 3 2 2 1 2

[0972] The three variables provide flexible control over the lists.Similarly, there are variables accessible via CCI to monitor the demandfor places in an event list (from queued events), and the events whichget into an event list (listed events).

[0973] Initially, max events is 1, max unified is 1 and max non unifiedis 1, as in the case of the first example in the table above. This givesbehaviour similar to that of Tardis 2.1, where only one event can bemade available to Monitors per update, and it is the first potentialevent in the event queue.

[0974] For an event that is received by the Tardis, it can “leave” theTardis in one of three ways:

[0975] Discarded—An event is discarded if it is never considered forplacing into an event list. This could be because an X Thread determinedit could discard the event, that is, not insert it in an event queue. Anevent is also discarded if it is placed in a queue, but subsequentchanges to the Slot's unify flag and a subsequent call to dear the queueout resulted in it being discarded.

[0976] Expired—The event made it into an event queue, but was removed bya Y Thread from the event queue because it did not meet the criteria toget into a read list and synthetic time passed it by (non unified).

[0977] Listed—The event made it into an event queue and into a read listand was made available to Monitors. Eventually it was cleared out of awrite list.

[0978] 4.3.1 Guaranteed Cell Pools

[0979] The Cell Pool holds a Guaranteed Cell Pool dedicated for eachSlot as well as the Shared Cell Pool, which it uses to store theincoming events and data. When a cell (event or data) is required for aSlot, the Slot's Guaranteed Cell Pool Manager is used. If the GuaranteedCell Pool Manager is unable to supply a cell (ie. it has no free cells),it attempts to get a cell from the Shared Cell Pool Manager.

[0980] The total number of cells allocated on start-up by the Cell Pool(Ntc) is given by the following formula:

Ntc=(Ngc*Ns)+Nsc where,

[0981] Ngc is the number of guaranteed cells per Slot, ie. perGuaranteed Cell Pool

[0982] Ns is the number of Slots, and

[0983] Nsc is the number of shared cells within the Shared Cell Pool.

[0984] The Shared Cell Pool and the Guaranteed Cell Pools behave in thesame way, they maintain a linked list of free cells and they have a lockfor accessing that list. Each cell has a Cell Pool Manager pointer sothat it can be returned to the appropriate Cell Pool Manager's free celllist.

[0985] Hence no entity in the Tardis needs to make a distinction betweena guaranteed cell and a shared cell

[0986] 5. Tardis Clock

[0987] A Tardis Clock is a process, which sends clock tick commands tothe Tardis' Synthetic Time CCI server. This action triggers an update inthe Tardis and provides the mechanism for the Tardis to move throughsynthetic time and make events available to Monitors. The rate at whichclock ticks are received by the Tardis in real time is the update ratein real time. It should be noted that if the Tardis' synthetic timewindow is less than the Tardis Clock's period, then it is possible thatthe Tardis' synthetic time could move ahead of real time.

[0988] 5.1 Clock Ticks

[0989] Clock ticks occur when a set of rules defined by a virtual FPGA(Field Programmable Gate Array) is satisfied. The inputs to the FPGA isa word in binary form, where each bit corresponds to the availability ofa clock event for that bit position.

[0990] The FPGA is shown in FIG. 29, with the table representing thefuse bits shown below along with the resulting clock tick expression:

[0991] tick=(A & C) or (A & B & C) or (C) or (A & B & C)

[0992] The fuse bits allow rules to be applied to the input word bits(A, B, C, . . . ) to determine whether a clock tick should occur. A fusebit of 1 means it is not blown and the relevant bit is input to therelevant AND gate. The results are combined by an OR gate. If a row offuse bits is not needed then the fuse bits should all be 0.

[0993] Table 2, of clock counters is also maintained, as is shown below.When a clock event with a certain ID is received, the clock event countfor that event is incremented. When a dock tick occurs, all clock eventcounters are decremented (but cannot be less than zero). A bit of theFPGA input word is formed if the corresponding counter is non zero:TABLE 2 Clock Event ID Clock Event counter FPGA Input word (1) 0 4 1 1 11 2 0 0 3 1 1

[0994] If each row of fuse bits is considered a binary word (W1, W2, W3,. . . ) then a rule will fail if:

[0995] rule fail=!I & W

[0996] So a tick should not occur when:

[0997] tick fail=(!I & W1) & (!I & W2) & (!I & W3)

[0998] Therefore a tick should occur when:

[0999] tick=!((!I & W1) & (!I & W2) & (!I & W3))

[1000] This can be evaluated very quickly. Note that since it is assumedthat the Tardis is built for a 64-bit architecture, we can allow for 64unique clock event IDs and as many rules as required. If we allow for nrules, the fuse bit table uses n 64 bit words.

[1001] Event IDs are allocated to clock event sources via CCI, which canalso be used as a mechanism to modify the FPGA fuse bit table and theclock event counters.

[1002] 6. Monitors

[1003] Monitors connect to the shared memory segment created by theTardis on start-up. This allows the Monitors to be able to read datafrom the Tardis Store, such as the read lists that have just beenprocessed by the Tardis. Note that they may use a Tardis Store Proxy todo this.

[1004] The Monitors need to wait until a switch has occurred, and theyneed to be able to detect a subsequent switch if one comes before theyfinish reading from the read list.

[1005] To do this, the Monitors wait for the update counter to changeindicating a switch. They then read all the data it requires from thearray, making local copies of data. It can verify the integrity of thedata by checking that the timestamp has not changed.

[1006] This is required every time data is read from the array. Even ifthe timestamp has not changed, if a pointer is then used to get data,the timestamp needs to be checked again to ensure that the pointerhasn't been de-referenced. This means that a Monitor should collect allthe data it needs from shared memory first, and then act on that dataonce its integrity has been verified.

[1007] There may be many different types of Monitors, but they need toget data from the Tardis in a similar way.

[1008] 7. Clients

[1009] 7.1 Overview

[1010] Clients communicate with the Tardis via Input Portal Connections.The Tardis' Z Threads almost continuously check for new Clients so theycan accept new Input Portal Connections.

[1011] Connections can be made through different Input Portals, so theTardis may have Clients sending events via sockets, and other paths,such as via shared memory.

[1012] The user and the Clients can request the number of availableInputs Portals, the type of available Input Portals, their identifiersand the details for available Input Portals from the Tardis via CCI, andthen establish connections on a specific Input Portal (as specified bytype and identifier). An identifier is preferably an ordered data valueassociated with the event by the Client. It may in a preferredembodiment be a integer within a range of natural numbers.

[1013] There may be many different types of Clients, but they need tosend data to the Tardis in a similar way.

Tardis Appendix/Glossary

[1014] A.1 Tardis

[1015] The Tardis is the event handling sub-system for Shapes Vector.The Tardis receives events from Tardis Clients and stores the events inshared memory for Tardis Monitors to read.

[1016] A.2 Tardis Monitor

[1017] Tardis Monitors are the event observation sub-systems for ShapesVector. They read and process the events made available for Monitors bythe Tardis.

[1018] A.3 Tardis Client

[1019] Tardis Clients connect to the Tardis and send events through anInput Portal Connection. The Input Portal can be of several differenttypes, such as a socket connection or shared memory.

[1020] A.4 Input Portal

[1021] An Input Portal is an object representing a conduit through whichevents are sent to the Tardis. Each Input Portal can have multiple InputPortal Connections that are specific connections through an Input Portalthrough which a single Client sends events to the Tardis. Each InputPortal has a type and an identifier.

[1022] A.5 Mutex

[1023] Mutexes are mutual exclusion locks that prevent multiple threadsfrom simultaneously executing critical sections of code, which accessshared data.

[1024] A.6 Semaphore

[1025] A semaphore is a non-negative integer count and is generally usedto coordinate access to resources. The initial semaphore count is set tothe number of free resources, then threads increment and decrement thecount as resources are added and removed. If the semaphore count dropsto zero, which means no available resources, threads attempting todecrement the semaphore will block until the count is greater than zero.

[1026] A.7 X Threads (Event Processing Threads)

[1027] X Threads are responsible for obtaining a new event from theInput Portal Connections and processing the event by storing it in theTardis Store. They also detect timed out Input Portal Connections.

[1028] A.8 Y Threads (Array Managing Threads)

[1029] Y Threads are responsible for updating the lists of events to beread by the Monitors. They do so by manipulating Slot and Event Cellindices for event queues. Y Threads are each responsible for updatingthe event queue for a specified range of Slots.

[1030] A.9 Z Threads

[1031] Z Threads are responsible for accepting new connection requestsfrom new Clients and creating new Input Portal Connections. These InputPortal Connections are added to a list, which is added to the M Thread'slist when the Z Threads are waiting.

[1032] A.10 Guarantee

[1033] Guarantees are a set of pre-allocated Event/Data Cells (createdupon start-up), used as the first choice of storage area for events anddata for each Slot.

Tardis Features Summary

[1034] TARDIS features specifically include:

[1035] 1. A set of slots where each semantic is associated with a uniqueslot. No slot is reused as the system evolves.

[1036] 2. A slot logic, which allows for flexible handling ofprioritised events.

[1037] 3. A synthetic dock which can be set to tick in a flexibleuser-specified manner.

[1038] 4. A taxonomy superimposed over the slots in order to group andcatalogue like semantics

[1039] It will be appreciated by those skilled in the art, that theinventions described herein are not restricted in their use to theparticular application described. Neither are the present inventionsrestricted in their preferred embodiments with regard to particularelements and/or features described or depicted herein. It will beappreciated that various modifications can be made without departingfrom the principles of these inventions. Therefore, the inventionsshould be understood to include all such modifications within theirscope. Part 1 SHAPES VECTOR 1 1 Shapes Vector Introduction 1 2Architectural Components 6  2.1 Primary Functional Architecture 6  2.1.1 Configuration Interface and I/O Sub-system 7   2.1.2 Sensors 8  2.1.3 Intelligent Agent Architecture 8  2.1.3.1 Knowledge Base 8 2.1.3.2 Intelligent Agents and Ontologies 9   2.1.4 The Tardis 10  2.1.5 Monitor 10  2.2 The Hardware 11  2.3 System Software 12 3 The“Classical” Visualisation Paradigm 13  3.1 Geo View 15  3.2 Data View 184 Intelligent Agents 23  4.1 Agent Architecture 23  4.2 InferencingStrategies 26   4.2.1 Traditional 26  4.2.2 Vectors 27  4.3 OtherApplications 31 5 Synthetic Stroboscopes and Selective Zoom 32  5.1Synthetic Strobes 32  5.2 Selective Zoom 34 6 Temporal Hierarchies 36 6.1 Strobes Revisited 36  6.2 User Information Overload 37  6.3 DataStreams and IA's 37 7 Other Visualisation Efforts 38  7.1 NetPARS 39 7.2 Security Visualisation 41  7.3 Network Visualisation 41  7.4 DataMining 42  7.5 Parentage and Autograph 43 Appendix Part 1- CustomControl Environments for Shapes Vector 44  A.1 Strategic Envirornment 45 A.2 Tactical Environment 45 PART 2 SHAPES VECTOR MASTER ARCHITECTURE47 1. Introduction 47  1.1 Shapes Vector Master Architecture 47  1.2Precis of this part of the specification 48 2. The Agent Architecture 49 2.1 Agent Architecture 50  2.2 A Note on the Tardis 53 3. InferencingStrategies 54  3.1 Traditional 54  3.2 Possiblistic 55  3.3 Vectors 56 3.4 Inferencing for Computer Security Applications 60  3.4 OtherApplications 63 4. Rules for Constructing an Agent 64 5. Agents and Time65  5.1 Data Streams and IA's 65  5.2 Temporal Event Mapping for Agents67 6. Implications for Higher Level Agents 68 7. Higher Level Ontologies69  7.1 Level 2 69   7.1.1 Relationships 70   7.1.2 InterrogativeOperators 72  7.2 Level 3 and Above 75  7.3 An Example of PossiblisticQuerying 76  7.4 An Example of the Use of Consistency 78 8. User Avatars79 9. Further Comments on the Architecture 80 10.1 AAFID 81 10.2Comparison with the Bass' Comments 82 11. A Multi-AbstractionalFramework for Shapes Vector Agents 83  11.1 Concepts 84 12. Summary 87Part 3 DATA VIEW SPECIFICATION 90 1. Data View Specification 90  1.1Universe 90  1.2 Objects 92  1.3 Aggregate Objects 93  1.4 ObjectSelector 95  1.5 Mass, Charge and Flavours 96  1.6 Forces 98  1.7Phantoming 99  1.8 Markers 99  1.9 Radius of Influence Display 100  1.10Pulses 101  1.11 Irregular Functions 101 Part 4 GEO VIEW SPECIFICATION102  1. Introduction 102  1.1 Identification 102  1.2 System Overview102   1.2.1 General 102   1.2.2 Geo View Module Scope 103  1.3 Overview103  2. Referenced Documents 104  2.1 Standard 104  3. Module-wideDesign Decisions 104  3.1 Design decisions and goals of Geo View 104  4.Module Architectural Design 105  4.1 Geo View Functional Design 106  4.1.1 Geo View General 106   4.1.1.1 Event Handling 106   4.1.1.2MasterTable Functionality 107   4.1.1.3 CCI Interface 107   4.1.1.4GeoView Processing and Caching 108   4.1.2 LayoutHierarchy 110   4.1.2.1Class Hierarchy 110   4.1.2.2 Logical Object Hierarchy 112   4.1.2.3Processing 114   4.1.3 Layout Structure Template Library (LSTL) 116  4.1.3.1 GenericGraph 117   4.1.3.2 GenericLine 117   4.1.3.3GenericMatrix 118   4.1.3.4 GenericRing 118   4.1.3.5 GenericStar 119  4.1.3.6 GenericRectangle 119   4.1.3.7 GenericTree 120  4.2 Concept ofExecution 120  4.3 Interface Design 120  5. Module Detailed Design 121 5.1 Geo View Classes 121   5.1.1 Geo View Class Summary 121  5.2LayoutHierarchy Classes 121   5.2.1 LayoutHierarchy Class Summary 121  5.2.2 Event Insertion and Removal Methods 123   5.2.2.1 cfInsert 123  5.2.2.2 Adding Inverse Attributes 123   5.2.2.3 NetworkObjectInsertion 124   5.2.2.4 Attribute Insertion 124   5.2.3 Layout RuleApplication Methods 125   5.2.3.1 applyLayoutstructureRulesToObject 125  5.2.3.2 applyAttachedToRulesToObject 127   5.2.3.3applyLocatedinRulesTo Object 128   5.2.3.4 findSatisfledRules 128  5.2.3.5 composeObjects 129   5.2.4 Leaf Edge Creation Methods 130  5.2.4.1 createEdges 130   5.2.4.2 createEdge 131   5.2.4.3 updateEdges131   5.2.4.4 addEdge 132   5.2.4.5 addGEdge 132  5.3LayoutStnictureTemplateLibrary (LSTL) Classes 132   5.3.1 LSTL TemplateClass Summary 132   5.3.2 GenericGraph Methods 133   5.3.2.1 NodeSeparation Factor 133   5.32.2 Rank Separation Factor 133   5.3.2.3Orientation 133   5.3.3 GenericLine Methods 133   5.3.3.1 Axis 133  5.3.3.2 Linear Direction 134   5.3.3.3 Origin 134   5.32.4 Separation134   5.3.4 GenericMatrix Methods 134   5.3.4.1 Width Separation 134  5.3.4.2 Depth Separation 134   5.3.4.3 Delete Policy 135   5.3.4.4Origin Policy 135   5.3.5 GenericRing Methods 135   5.35.1 AngularDirection 135   5.3.5.2 Radius 135   5.3.5.3 Separation 136   5.3.6GenericStar Methods 136   52.6.1 Root Height 136   5.3.7GenericRectangle Methods 136   5.3.7.1 Angular Direction 136   5.3.7.2Start Side 136   5.3.7.3 Width Separation 137   5.3.7.4 Depth Separation137   5.3.7.5 Width 137   5.3.7.6 Depth 137   5.3.8 LSTL Class Interface137   5.3.8.1 Memory Allocation 138   5.3.8.2 Relative Placement 138  5.3.9 T Interface 138  6. Appendix for GeoView 139  7.1 Abbreviations139  7.2 Definition of Terms 140 Part 5 TARDIS SPECIFICATION 142 1.Tardis Specification 142  1.1 Introduction 142  1.2 Assumptions 143 2.Overview of the Tardis 143  2.1 Components 144  2.2 Overview ofOperation 145 3. Tardis Concepts 145  3.1 Events 145  3.2 TimeStamp 147 3.3 Shared Memory 148  3.4 Time 148  3.4.1 Universal Time 148  3.4.2Synthetic Time 148  3.5 Process and Thread Activity 149 4. FunctionalOverview of the Tardis 150  4.1 Tardis Threads 150  4.2 Tardis Operation152  4.3 Tardis Store 154   4.3.1 Guaranteed Cell Pools 156 5. TardisClock 157  5.1 Clock Ticks 158 6. Monitors 159 7. Clients 160  7.1Overview 160 Tardis Appendix/Glossary 161  A.1 Tardis 161  A.2 TardisMonitor 161  A.3 Tardis Client 161  A.4 Input Portal 161  A.5 Mutex 162 A.6 Semaphore 162  A.7 X Threads (Event Processing Threads) 162  A.8 YThreads (Array Managing Threads) 162  A.9 Z Threads 163  A.10 Guarantee163  Tardis Features Summary 163 Claims defining the invention are asfollows: 170

The claims defining the invention are as follows:
 1. A system for creating human observable objects in a modelling system consists of: one or more representations of real or virtual objects; a virtual universe; a mapping from a representation of a real or virtual object to a virtual object having at least one of visual, aural or haptic attributes wherein the position of a said virtual object in said universe is in accordance with layout rules, each rule expressed in terms of the representations of first and second real or virtual objects and the interrelationship between said first and second real or virtual objects and a related position determination.
 2. A system according to claim 1 wherein said universe is two or three-dimensional.
 3. A system according to claim 1 wherein said human can observe said universe from any position with any orientation.
 4. A system according to claim 1 wherein a virtual object can be deleted or changed.
 5. A system according to claim 1 wherein a virtual object is added.
 6. A system according to claim 1 wherein said mapping is changed.
 7. A system according to claim 1 wherein virtual objects are grouped.
 8. A system according to claim 1 wherein a human temporarily changes the attributes or position of one or more virtual objects.
 9. A system according to claim 1 wherein a function determined by a human determines a set of selected real or virtual objects that have a corresponding virtual object's attributes or position changed.
 10. A system according to claim 9 wherein said human controls the degree of change.
 11. A system according to claims 9 or 10 wherein a further function determined by a human determines a subset of said selected real or virtual objects that have a corresponding virtual object's attributes changed.
 12. A system according to claim 1 wherein a human temporarily changes the attributes or position of one or more virtual objects for a predetermined period.
 13. A system according to claim 9 wherein a corresponding virtual object's attributes or position are changed for a predetermined period.
 14. A system according to claims 12 or 13 wherein said period begins when a predetermined event occurs in said universe.
 15. A system according to claims 12 or 13 wherein said period begins at a predetermined time.
 16. A system according to claims 12 or 13 wherein said period begins periodically.
 17. A system according to claim 16 wherein said periodicity is changed by said human.
 18. A system according to claim 1 wherein a human determines the position of a virtual object.
 19. A system according to claim 1 wherein a human determines a set of real or virtual objects for which a predetermined layout rule applies.
 20. A system according to claim 19 wherein said layout rule determines the relative position of virtual objects of said set to each other.
 21. A system according to claim 1 wherein a human chooses an inter-relationship and all real or virtual objects having that relationship are positioned relative to each other according to a predetermined layout rule.
 22. A system according to claim 20 wherein said layout rule determines the scale of representation of said one of more virtual objects relative to each other.
 23. A system according to claim 1 wherein a human determines the real or virtual object represented as a virtual object that has at least one of visual, aural or haptic attributes or position determined by applying a predetermined time related function.
 24. A system according to claim 23 wherein said virtual object exists in said universe for a predetermined time period.
 25. A system according to claim 1 wherein said mapping determines an attribute value according to the first attribute, condition pair of an ordered list of pairs where the condition is satisfied by the representation of the real or virtual object.
 26. A system according to claim 1 where there exists more that one possible position for a said virtual object and a priority rule determines the position of said virtual object in the universe. 